Example 6: Counting and measuring freshwater snails

In this example we will use thresholding and watershed algorithms to count freshwater snails.

Input - Snails photographed from a camera stand. Variable brightness across the tray and snail clumping are the biggest challenges.

Results - After applying adaptive thresholding and a watershed algorithm, the snail separate well from the background. Now we can count them, and measure size, shape and colouration

Low throughput

First, we test a single image with the low throughput workflow.

import phenopype as pp
import os

image_dir = "images/snails1.jpg"

ct = pp.load_image(image_dir,
                   cont=True,  # load as container
                   dirpath= "../_temp/output/ex6",  # specify save-directory
                   save_suffix = "snails1") # give files a save-suffix (e.g. "contours_ex6.csv")

Directory to save files set at - E:\git_repos\phenopype\_temp\output\ex6

The load method of a container attempts to load any previously saved results, masks, etc. from the specified dirpath. reset will only reset the modified images, as well as image and contour DataFrames. This will preserve any drawn masks or other data stemming from user interaction.

ct.load()
ct.reset()

AUTOLOAD
- masks_snails1.csv

Begin by drawing a mask around the snails inside the tray by dragging a rectangle around them - finish with Enter, abort with Esc.

pp.preprocessing.create_mask(ct)

- create mask

Now we need to measure the pixel-to-mm-ratio. The snail pictures don’t contain a scale that is appropriate for automatic detection - the millimeter-paper on the side does not contain enough unique keypoints to be detected by the AKAZE algorithm that powers find_scale. Therefore each image needs to be manually measured by dragging a line across the mm-paper, and entering the distance.

Note: It is better to measure a long distance than a short distance to minimize the measurement error - use the full length (~70 mm) here.

pp.preprocessing.create_scale(ct)

- measure pixel-to-mm-ratio
Scale set
- add column length

For high data quality it is important to verify the ID of the specimen in the current picture. Often, the picture name contains the ID, but typically an label is placed inside the image. Using the enter_data tool we open the image and an entry prompt that will create a column with a name of our chosing inside all exported results.

pp.preprocessing.enter_data(ct, columns="ID")

- add column ID

Next is segmentation. First we blur the image a little, convert it to a binary image, and look at the results:

pp.segmentation.blur(ct)
pp.segmentation.threshold(ct,
                          method="adaptive",
                          blocksize=59, ## relatively low sensitivity
                          constant=10) ## relatively constant gets subtracted from the binary result
pp.show_image(ct) # `ct` automatically shows the last edit within the container, `ct.image` would also work

- include mask "mask1" pixels

The break up the clumping, we apply the watershed algorithm to the binarized image:

pp.segmentation.watershed(ct, distance_cutoff=0.5) # , iterations=1, kernel_size=1
pp.show_image(ct)

With this image we can find the contours - for watershed we have to select the "ccomp" option.

pp.segmentation.find_contours(ct,
                              retrieval="ccomp", # this finds the splitted inner rather than the outer contours
                              min_area=200,  # noise removal
                              subset="child") # needs to be "child" for watershed

With the contours we have acquired the primary information (counts, size, area) contained in the image. However, since we have the contour location, we get the colour for free. Note that we also measure the background whiteness of each scale (background=True). This is important, as the snails lie across a brightness gradient within the images.

pp.measurement.colour_intensity(ct,
                                background=True) # this measures the whiteness of the area around each detected snails

Now we draw the contours…

pp.visualization.select_canvas(ct, # onto which image should the contours be draw
                               canvas="raw") # raw = original image
pp.visualization.draw_contours(ct,
                               fill=0,
                               line_width=2,
                               watershed=True, # this flag needs to be added when using watershedding
                               bounding_box=True) # this indicates the area where the background was measured
pp.show_image(ct.canvas)

- raw image

… and save them, as well as the masks (if we need to redo this) and the canvas for quality control.

pp.export.save_canvas(ct)
pp.export.save_masks(ct)
pp.export.save_contours(ct)

- canvas saved under ../_temp/output/ex6\canvas_snails1.jpg (overwritten).
- masks saved under ../_temp/output/ex6\masks_snails1.csv (overwritten).
- contours saved under ../_temp/output/ex6\contours_snails1.csv (overwritten).

Note: the countour csv contains only the inner, watershedded contours (separated=“child”), the outer (unseparated=“parent”) were removed with find_contours.

High throughput

As for the other examples I have created a preset (ex6) with appropriate settings for the example. The template can be passed to the pype using config_preset="ex6" - see below.

Because there is a brightness gradient across the image, correcting the exposure across all images will not work. Instead we use the colour_intensity to return the local background whiteness (within the rectangle, excluding all snail pixels). Each snail will have an individual background brightness score that can be used to normalize the detected values, so that the colour intensity becomes a meaningful trait within and across images.

First, set some directories and inspect the preset.

import phenopype as pp
import os

project_root = r"../_temp/output/ex6_proj"
images = r"images"

print(pp.presets.ex6)

preprocessing:
- create_mask
- create_scale
- enter_data
segmentation:
- blur:
    kernel_size: 3
- threshold:
    method: adaptive
    blocksize: 59
    constant: 10
    channel: gray
- watershed:
    distance_cutoff: 0.5
# - draw  # to separate snails
- find_contours:
    retrieval: ccomp # needs to be ccomp for watershed
    min_diameter: 0
    min_area: 200
    subset: child # needs to be child for watershed
measurement:
- colour_intensity:
    background: True
visualization:
- select_canvas:
    canvas: raw
- draw_contours:
    line_width: 2
    label_width: 1
    label_size: 1
    fill: 0
    watershed: true
    bounding_box: True
- draw_masks
export:
- save_contours:
    save_coords: False
- save_colours
- save_masks

Then create a project. As stated above in the low throughput instructions: defining a project wide scale will not work because the provided reference card is too homogeneous, i.e. does not contain enough keypoints for registration by find_scale. If you already have created the project, simply load it.

if not os.path.isdir(project_root):
    proj = pp.project(root_dir=project_root, overwrite=True)
    proj.add_files(image_dir=images, include="snails", overwrite=True)
    proj.add_config(name = "v1", overwrite=True, preset="ex6")
    pp.project.save(proj, overwrite=True)
else:
    proj = pp.project.load(project_root)

--------------------------------------------
Project loaded from
E:\git_repos\phenopype\_temp\output\ex6_proj
--------------------------------------------

Now you can loop through all directories stored within the project object to detect the snails.

for i in proj.dirpaths:
    p = pp.pype(i, name="v1")

E:\git_repos\phenopype\_temp\output\ex6_proj\data\0__snails1\pype_config_v1.yaml


------------+++ new pype iteration 2020:05:01 18:48:25 +++--------------


AUTOLOAD
- columns ID from attributes.yaml
- current scale information loaded from attributes.yaml
- masks_v1.csv
PREPROCESSING
create_mask
- mask with label mask1 already created (overwrite=False)
create_scale
- scale pixel-to-mm-ratio already measured (overwrite=False)
enter_data
- column ID already created (overwrite=False)
SEGMENTATION
blur
threshold
- include mask "mask1" pixels
watershed
find_contours
MEASUREMENT
colour_intensity
VISUALIZATION
select_canvas
- raw image
draw_contours
draw_masks
 - show mask: mask1.
EXPORT
save_contours
- contours saved under E:\git_repos\phenopype\_temp\output\ex6_proj\data\0__snails1\contours_v1.csv (overwritten).
save_colours
- colours saved under E:\git_repos\phenopype\_temp\output\ex6_proj\data\0__snails1\colours_v1.csv (overwritten).
save_masks
- masks saved under E:\git_repos\phenopype\_temp\output\ex6_proj\data\0__snails1\masks_v1.csv (overwritten).
AUTOSAVE
save_canvas
- canvas saved under E:\git_repos\phenopype\_temp\output\ex6_proj\data/0__snails1\canvas_v1.jpg (overwritten).
save_data_entry
- column ID not saved (overwrite=False)
save_scale
- save scale to attributes (overwriting)


TERMINATE
E:\git_repos\phenopype\_temp\output\ex6_proj\data\0__snails2\pype_config_v1.yaml


------------+++ new pype iteration 2020:05:01 18:48:27 +++--------------


AUTOLOAD
- columns ID from attributes.yaml
- current scale information loaded from attributes.yaml
- masks_v1.csv
PREPROCESSING
create_mask
- mask with label mask1 already created (overwrite=False)
create_scale
- scale pixel-to-mm-ratio already measured (overwrite=False)
enter_data
- column ID already created (overwrite=False)
SEGMENTATION
blur
threshold
- include mask "mask1" pixels
watershed
find_contours
MEASUREMENT
colour_intensity
VISUALIZATION
select_canvas
- raw image
draw_contours
draw_masks
 - show mask: mask1.
EXPORT
save_contours
- contours saved under E:\git_repos\phenopype\_temp\output\ex6_proj\data\0__snails2\contours_v1.csv (overwritten).
save_colours
- colours saved under E:\git_repos\phenopype\_temp\output\ex6_proj\data\0__snails2\colours_v1.csv (overwritten).
save_masks
- masks saved under E:\git_repos\phenopype\_temp\output\ex6_proj\data\0__snails2\masks_v1.csv (overwritten).
AUTOSAVE
save_canvas
- canvas saved under E:\git_repos\phenopype\_temp\output\ex6_proj\data\0__snails2\canvas_v1.jpg (overwritten).
save_data_entry
- column ID not saved (overwrite=False)
save_scale
- save scale to attributes (overwriting)


TERMINATE

In the end, use the collect_results method of proj to save all results to a folder in the root directory.

proj.collect_results(name="v1",files=["contours"], folder="contours_ref", overwrite=True)

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-1-b2cf852f4471> in <module>
----> 1 proj.collect_results(name="v1",files=["contours"], folder="contours_ref", overwrite=True)

NameError: name 'proj' is not defined