API reference

Projects

Phenopype projects are composed of a directory tree in which each folder contains the copy or a link to a single raw image file. Each raw should have only one folder, so that result or intermediate output from different iterations of processing and analysis are stored side by side. The project root folder should be separate from the raw data, e.g. as a folder inside of your project folder:

import phenopype as pp

myproj = pp.project(root_dir="\my_project\phenopype")
myproj.add_files(image_dir="\my_project\data_raw")
myproj.add_config(name = "v1")
pp.project.save(myproj)

## after closing Python, the project can be accessed using the projects load function
myproj = pp.project.load("\my_project\phenopype\project.data")

Above code creates a folder structure as follows:

my_project\
 data_raw
 phenopype\             # create phenopype project here with "pp.project"
  data\                 # created automatically
   project.data         # created with "pp.project.save"
   file1\               # add files to project using "pp.project.add_files"
    raw.jpg                     # created by "pp.project.add_files"
    attributes.yaml     # created by "pp.project.add_files"
    pype_config_v1.yaml # added with "pp.project.add_config"
    results_v1.csv
   file2\
    ...
   ...
 data
 scripts
 manuscript
 figures

Important

Currently the only useful information contained in the project object (myproj) is a list of all directories inside the project’s directory tree. It is important to save both the project AND all results in the data directories using the appropriate functions in (Export). Future release will expand the functionality of the project class and its associated methods.

class phenopype.main.project(root_dir, overwrite=False)

Bases: object

Initialize a phenopype project with a root directory path. Phenopype will create the project folder at the provided location.

Parameters
  • rootdir (str) – path to root directory of the project where folder gets created

  • overwrite (bool, optional) – overwrite option, if a given root directory already exist (WARNING: also removes all folders inside)

Returns

project – phenopype project object

Return type

project

add_files(image_dir, filetypes=['jpg', 'JPG', 'jpeg', 'JPEG', 'tif', 'png'], include=[], exclude=[], raw_mode='copy', search_mode='dir', unique_mode='path', overwrite=False, resize=1, **kwargs)

Add files to your project from a directory, can look recursively. Specify in- or exclude arguments, filetypes, duplicate-action and copy or link raw files to save memory on the harddrive. For each found image, a folder will be created in the “data” folder within the projects root directory. If found images are in subfolders and search_mode is recursive, the respective phenopype directories will be created with flattened path as prefix.

E.g., with “raw_files” as folder with the original image files and “phenopype_proj” as rootfolder:

  • raw_files/file.jpg ==> phenopype_proj/data/file.jpg

  • raw_files/subdir1/file.jpg ==> phenopype_proj/data/1__subdir1__file.jpg

  • raw_files/subdir1/subdir2/file.jpg ==> phenopype_proj/data/2__subdir1__subdir2__file.jpg

Parameters
  • image_dir (str) – path to directory with images

  • filetypes (list or str, optional) – single or multiple string patterns to target files with certain endings. “default_filetypes” are configured in settings.py

  • include (list or str, optional) – single or multiple string patterns to target certain files to include

  • exclude (list or str, optional) – single or multiple string patterns to target certain files to exclude - can overrule “include”

  • raw_mode ({"copy", "link"} str, optional) – how should the raw files be passed on to the phenopype directory tree: “copy” will make a copy of the original file, “link” will only send the link to the original raw file to attributes, but not copy the actual file (useful for big files)

  • search_mode ({"dir", "recursive"}, str, optional) – “dir” searches current directory for valid files; “recursive” walks through all subdirectories

  • unique_mode ({"filepath", "filename"}, str, optional:) – how to deal with image duplicates - “filepath” is useful if identically named files exist in different subfolders (folder structure will be collapsed and goes into the filename), whereas filename will ignore all similar named files after their first occurrence.

  • kwargs – developer options

add_config(name, config_preset=None, interactive=False, overwrite=False, idx=0, **kwargs)

Add pype configuration presets to all image folders in the project, either by using the templates included in the presets folder, or by adding your own templates by providing a path to a yaml file. Can be tested and modified using the interactive flag before distributing the config files.

Parameters
  • name (str) – name of config-file. this gets appended to all files and serves as and identifier of a specific analysis pipeline

  • preset (str, optional) – can be either a string denoting a template name (e.g. preset1, preset2, landamarking1, … - in “phenopype/settings/presets.py”) or a path to a compatible yaml file

  • interactive (bool, optional) – start a pype and modify preset before saving it to phenopype directories

  • overwrite (bool, optional) – overwrite option, if a given pype config-file already exist

  • kwargs – developer options

add_scale(reference_image, overwrite=False, **kwargs)

Add pype configuration presets to all project directories.

Parameters
  • reference_image (str) – name of template image, either project directory or file link. template image gets stored in root directory, and information appended to all attributes files in the project directories

  • overwrite (bool, optional) – overwrite option, if a given pype config-file already exist

  • template (bool, optional) – should a template for scale detection be created. with an existing template, phenopype can try to find a reference card in a given image, measure its dimensions, and adjust pixel-to-mm-ratio and colour space

collect_results(name, files=None, folder='results', overwrite=False)

Collect results (images or CSVs) from the entire data folder. Search by pype name (e.g. “v1”) and filter by filetypes (e.g. landmarks, contours or colours)

Parameters
  • name (str) – name of the pype or save_suffix

  • files (str or list, optional) – filetypes to look for (e.g. landmarks, contours or colours)

  • folder (str, optional) – folder in the root directory where the results are stored

  • overwrite (bool, optional) – should the results be overwritten

edit_config(name, step, function, **kwargs)

[new/experimental] Add or edit functions in all configuration files of a project.

Parameters
  • name (str) – name of config-file. this gets appended to all files and serves as and identifier of a specific analysis pipeline

  • step (str) – name of the step the function is in

  • function (str) – name of the function

static save(project, overwrite=False)

Save project to root directory

Parameters
  • project (phenopype.main.project) – save project file to root dir of project (saves ONLY the python object needed to call file-lists, NOT collected data), which needs to be saved separately with the appropriate export functions (e.g. phenopype.export.save_contours() or phenopype.export.save_canvas())

  • overwrite (bool, optional) – should existing project data be overwritten

static load(path)

Load phenoype project.data file to Python namespace.

Parameters

path (str) – path to project.data, or the directory project.data is contained in (root_dir)

Returns

project – phenopype project object

Return type

project

pype (high throughput function)

Method used to process image datasets with high throughput and reproducibility. Should be used in conjunction with image organized in a Phenopype directory structure. To get started with the high througput workflow, see Tutorial 2 and Tutorial 3.
class phenopype.main.pype(image, name, config_preset=None, config_location=None, dirpath=None, skip=False, feedback=True, delay=100, **kwargs)

Bases: object

The pype is phenopype’s core method that allows running all functions that are available in the program’s library in sequence. Users can execute the pype method on a filepath, an array, or a phenopype directory, which always will trigger three actions:

  1. open the contained yaml configuration with the default OS text editor

  2. parse the contained functions and execute them in the sequence (exceptions will be passed, but returned for diagnostics)

  3. open a Python-window showing the processed image.

After one iteration of these steps, users can evaluate the results and decide to modify the opened configuration file (e.g. either change function parameters or add new functions), and run the pype again, or to terminate the pype and save all results. The processed image, any extracted phenotypic information, as well as the modified config-file is stored inside the image directory, or a user-specified directory. By providing unique names, users can store different pype configurations and the associated results side by side.

Parameters
  • image (array or str) – can be either a numpy array or a string that provides the path to source image file or path to a valid phenopype directory

  • name (str) – name of pype-config - will be appended to all results files

  • config_preset (str, optional) – chose from given presets in phenopype/settings/pype_presets.py (e.g. preset1, preset2, …)

  • config_location (str, optional) – custom path to a pype template (needs to adhere to yaml syntax and phenopype structure)

  • delay (int, optional) – time in ms to add between reload attemps of yaml monitor. increase this value if saved changes in config file are not parsed in the first attempt.

  • dirpath (str, optional) – path to an existing directory where all output should be stored

  • skip (bool, optional) – skip directories that already have “name” as a suffix in the filename

  • feedback (bool, optional) – don’t open text editor or window, just apply functions and terminate

  • kwargs – developer options

Utility functions

Utility functions, e.g. to load and display images ,e.g. pp.utils.load_image() and pp.utils.show_image(). Any of these functions can be called directly after loading phenopype without the need to add the utils submodule:

import phenopype as pp

image_path = "image_dir/my_image.jpg"

image = pp.load_image(image_path)       # instead of pp.utils.load_image
pp.show_image(image)                    # instead of pp.utils.show_image
dir(pp)                                 # shows available classes and functions
class phenopype.utils.container(image, df_image_data, dirpath=None, save_suffix=None)

Bases: object

A phenopype container is a Python class where loaded images, dataframes, detected contours, intermediate output, etc. are stored so that they are available for inspection or storage at the end of the analysis. The advantage of using containers is that they don’t litter the global environment and namespace, while still containing all intermediate steps (e.g. binary masks or contour DataFrames). Containers can be used manually to analyse images, but typically they are created dynamically within the pype-routine.

Parameters
  • image (ndarray) – single or multi-channel iamge as an array (can be created using load_image or load_directory).

  • df_image_data (DataFrame) – a dataframe that contains meta-data of the provided image to be passed on to all results-DataFrames

  • save_suffix (str, optional) – suffix to append to filename of results files

load(dirpath=None, save_suffix=None, contours=False, **kwargs)

Autoload function for container: loads results files with given save_suffix into the container. Can be used manually, but is typically used within the pype routine.

Parameters

save_suffix (str, optional) – suffix to include when looking for files to load

reset()

Resets modified images, canvas and df_image_data to original state. Can be used manually, but is typically used within the pype routine.

save(dirpath=None, overwrite=False, **kwargs)

Autosave function for container.

Parameters
  • dirpath (str, optional) – provide a custom directory where files should be save - overwrites dirpath provided from container, if applicable

  • export_list (list, optional) – used in pype rountine to check against already performed saving operations. running container.save() with an empty export_list will assumed that nothing has been saved so far, and will try

  • overwrite (bool, optional) – gloabl overwrite flag in case file exists

phenopype.utils.load_directory(directory_path, cont=True, df=True, meta=True, resize=1, save_suffix=None, **kwargs)
Parameters
  • directory_path (str or ndarray) – path to a phenopype project directory containing raw image, attributes file, masks files, results df, etc.

  • cont (bool, optional) – should the loaded image (and DataFrame) be returned as a phenopype container

  • df (bool, optional) – should a DataFrame containing image information (e.g. dimensions) be returned.

  • meta (bool, optional) – should the DataFrame encompass image meta data (e.g. from exif-data). This works only when obj_input is a path string to the original file.

  • resize (float, optional) – resize factor for the image (1 = 100%, 0.5 = 50%, 0.1 = 10% of original size).

  • save_suffix (str, optional) – suffix to append to filename of results files

  • kwargs – developer options

Returns

A phenopype container is a Python class where loaded images, dataframes, detected contours, intermediate output, etc. are stored so that they are available for inspection or storage at the end of the analysis.

Return type

container

phenopype.utils.load_image(obj_input, mode='default', cont=False, df=False, dirpath=None, meta=False, resize=1, save_suffix=None, **kwargs)

Create ndarray from image path or return or resize exising array.

Parameters
  • obj_input (str or ndarray) – can be a path to an image stored on the harddrive OR an array already loaded to Python.

  • mode ({"default", "colour","gray"} str, optional) –

    image conversion on loading:
    • default: return image as is

    • colour: convert image to 3channel bgr

    • gray: convert image to single channel

  • cont (bool, optional) – should the loaded image (and DataFrame) be returned as a phenopype container

  • df (bool, optional) – should a DataFrame containing image information (e.g. dimensions) be returned.

  • dirpath (str, optional) – path to an existing directory where all output should be stored

  • meta (bool, optional) – should the DataFrame encompass image meta data (e.g. from exif-data). This works only when obj_input is a path string to the original file.

  • resize (float, optional) – resize factor for the image (1 = 100%, 0.5 = 50%, 0.1 = 10% of original size).

  • save_suffix (str, optional) – suffix to append to filename of results files, if container is created

  • kwargs – developer options

Returns

  • ct (container) – A phenopype container is a Python class where loaded images, dataframes, detected contours, intermediate output, etc. are stored so that they are available for inspection or storage at the end of the analysis.

  • image (ndarray) – original image (resized, if selected)

  • df_image_data (DataFrame) – contains image data (+meta data), if selected

phenopype.utils.load_image_data(obj_input, resize=1)

Create a DataFreame with image information (e.g. dimensions).

Parameters
  • obj_input (str or ndarray) – can be a path to an image stored on the harddrive OR an array already loaded to Python.

  • resize (float) – resize factor for the image (1 = 100%, 0.5 = 50%, 0.1 = 10% of original size). gets stored to a DataFrame column

Returns

image_data – contains image data (+meta data, if selected)

Return type

dict

phenopype.utils.load_meta_data(image_path, show_fields=False, fields=['DateTimeOriginal', 'Model', 'LensModel', 'ExposureTime', 'ISOSpeedRatings', 'FNumber'])

Extracts metadata (mostly Exif) from original image

Parameters
  • image_path (str) – path to image on harddrive

  • show_fields (bool, optional) – show which exif data fields are available from given image

  • fields (list or str) – which exif data fields should be extracted. default fields can be modified in settings

Returns

exif_data – image meta data (exif)

Return type

dict

phenopype.utils.show_image(image, max_dim=1200, position_reset=True, position_offset=25, window_aspect='free', check=True, **kwargs)

Show one or multiple images by providing path string or array or list of either.

Parameters
  • image (array, list of arrays) – the image or list of images to be displayed. can be array-type, or list or arrays

  • max_dim (int, optional) – maximum dimension on either acis

  • window_aspect ({"fixed", "free"} str, optional) – type of opencv window (“free” is resizeable)

  • position_reset (bool, optional) – flag whether image positions should be reset when reopening list of images

  • position_offset (int, optional) – if image is list, the distance in pixels betweeen the positions of each newly opened window (only works in conjunction with “position_reset”)

  • check (bool, optional) – user input required when more than 10 images are opened at the same time

phenopype.utils.save_image(image, name, dirpath='E:\\git_repos\\phenopype', resize=1, append='', extension='jpg', overwrite=False, **kwargs)

Save an image (array) to jpg.

Parameters
  • image (array) – image to save

  • name (str) – name for saved image

  • save_dir (str, optional) – directory to save image

  • append (str, optional) – append image name with string to prevent overwriting

  • extension (str, optional) – file extension to save image as

  • overwrite (boo, optional) – overwrite images if name exists

  • resize (float, optional) – resize factor for the image (1 = 100%, 0.5 = 50%, 0.1 = 10% of original size).

  • kwargs – developer options

Image analysis (core modules)

Core image processing functions. Any function can be executed in prototyping, low throughput, and high throughput workflow. In general, the order in which the different steps are performed should always be the same:

preprocessing > segmentation > measurement > visualization > export

In general, any function can either take an array or a phenopype container. If an array is passed, additional input arguments may be required (e.g. to draw contours onto an image, but an array and a DataFrame containing the contours must be supplied, whereas a container already includes both).

Preprocessing

phenopype.core.preprocessing.create_mask(obj_input, df_image_data=None, include=True, label='mask1', overwrite=False, tool='rectangle', **kwargs)

Draw rectangle or polygon mask onto image by clicking and dragging the cursor over the image. One mask can contain multiple sets of coordinates, i.e. multiple and not overallping masks. For rectangle mask, finish with ENTER. For polygons, finish current polygon with CTRL, and then with ENTER.

Parameters
  • obj_input (array or container) – input object

  • df_image_data (DataFrame, optional) – an existing DataFrame containing image metadata, will be added to mask output DataFrame

  • include (bool, optional) – determine whether resulting mask is to include or exclude objects within

  • label (str, optinal) – assigns a label to the mask

  • overwrite (bool, optional) – if working using a container, or from a phenopype project directory, should existing masks with the same label be overwritten

  • tool ({"rectangle", "polygon"} str, optional) – select tool by which mask is drawn

Returns

df_masks – contains mask coordiantes

Return type

DataFrame or container

phenopype.core.preprocessing.create_scale(obj_input, df_image_data=None, df_masks=None, mask=False, overwrite=False, template=False, **kwargs)

Measure a size or colour reference card. Minimum input interaction is measuring a size reference: click on two points inside the provided image, and enter the distance - returns the pixel-to-mm-ratio as integer or inserts it into a provided DataFrame (df_image_data). In an optional second step, drag a rectangle mask over the reference card to exclude it from any subsequent image segementation. The mask is exported as new DataFrame, OR, if provided before, gets appended to an existing one (df_masks). The mask can also be stored as a template for automatic scale detection with the “find_scale” function.

Parameters
  • obj_input (array or container) – input object

  • df_image_data (DataFrame, optional) – an existing DataFrame containing image metadata to add the scale information to (pixel-to-mm-ratio)

  • df_masks (DataFrame, optional) – an existing DataFrame containing masks to add the created mask to

  • mask (bool, optional) – mask a reference card inside the image (returns a mask DataFrame)

  • overwrite (bool, optional) – if a container is supplied, or when working from the pype, should any exsting scale information (px-to-mm-ratio) or template be overwritten

  • template (bool, optional) – should a template for scale detection be created. with an existing template, phenopype can try to find a reference card in a given image, measure its dimensions, and adjust and colour space. automatically creates and returns a mask DataFrame that can be added to an existing one

  • kwargs (optional) – developer options

Returns

  • px_mm_ratio (int or container) – pixel to mm ratio - not returned if df_image_data is supplied

  • df_image_data (DataFrame or container) – new or updated, containes scale information

  • df_masks (DataFrame or container) – new or updated, contains mask information

  • template (array or container) – template for reference card detection

phenopype.core.preprocessing.enter_data(obj_input, df_image_data=None, columns='ID', overwrite=False, label_size='auto', label_width='auto', label_colour='red', **kwargs)

Generic data entry that can be added as columns to an existing DataFrame. Useful for images containing labels, or other comments.

Parameters
  • obj_input (array or container) – input object

  • df_image_data (DataFrame, optional) – an existing DataFrame containing image metadata to add columns to

  • columns (str or list) – columns to be added to existing or generic data frame

  • overwrite (bool, optional) – overwrite existing columns in df

  • fontsize (int, optional) – fonsize for onscreen display.

  • font_col ({"red", "green", "blue", "black", "white"} str, optional) – font colour for onscreen display.

Returns

df_other_data – contains the entered data

Return type

DataFrame or container

phenopype.core.preprocessing.find_scale(obj_input, df_image_data=None, template=None, overwrite=False, equalize=False, min_matches=10, resize=1, px_mm_ratio_ref=None, df_masks=None)

Find scale from a template created with “create_scale”. Image registration is run by the “AKAZE” algorithm. Future implementations will include more algorithms to select from. First, use “create_scale” with “template=True” and pass the template to this function. This happends automatically in the low and high throughput workflow (i.e., when “obj_input” is a container, the template image is contained within. Use “equalize=True” to adjust the histograms of all colour channels to the reference image.

AKAZE: http://www.bmva.org/bmvc/2013/Papers/paper0013/abstract0013.pdf

Parameters
  • obj_input (array or container) – input for processing

  • df_image_data (DataFrame, optional) – an existing DataFrame containing image metadata to add the scale information to (pixel-to-mm-ratio)

  • df_masks (DataFrame, optional) – an existing DataFrame containing masks to add the detected mask to

  • template (array or container, optional) – reference image of scale

  • equalize (bool, optional) – should the provided image be colour corrected to match the template images’ histogram

  • min_matches (int, optional) – minimum key point matches for image registration

  • resize (num, optional) – resize image to speed up detection process. default: 0.5 for images with diameter > 5000px (WARNING: too low values may result in poor detection performance or even crashes)

  • overwrite (bool, optional) – overwrite existing scale_current_px_mm_ratio in container

  • px_mm_ratio_ref (int, optional) – pixel-to-mm-ratio of the template image

Returns

  • scale_current_px_mm_ratio (int or container) – pixel to mm ratio of current image

  • image (array or container) – if scale contains colour information, this is the corrected image

  • df_masks (DataFrame or container) – contains mask coordinates to mask reference card within image from segmentation algorithms

phenopype.core.preprocessing.invert_image(obj_input)

Invert all pixel intensities in image (e.g. 0 to 255 or 100 to 155)

Parameters

obj_input (array or container) – input for processing

Returns

image – inverted image

Return type

array or container

phenopype.core.preprocessing.resize_image(obj_input, factor=1)

Resize image by resize factor

Parameters
  • obj_input (array or container) – input for processing

  • resize (float, optional) – resize factor for the image (1 = 100%, 0.5 = 50%, 0.1 = 10% of original size).

Returns

image – resized image

Return type

array or container

Segmentation

phenopype.core.segmentation.blur(obj_input, kernel_size=5, method='averaging', sigma_color=75, sigma_space=75)

Apply a blurring algorithm to the image.

Parameters
  • obj_input (array or container) – input object

  • kernel_size (int, optional) – size of the blurring kernel (has to be odd - even numbers will be ceiled)

  • method ({averaging, gaussian, median, bilateral} str, optional) – blurring algorithm

  • sigma_colour (int, optional) – for ‘bilateral’

  • sigma_space (int, optional) – for ‘bilateral’

Returns

image – blurred image

Return type

array or container

phenopype.core.segmentation.draw(obj_input, overwrite=False, tool='line', line_colour='black', line_width='auto', **kwargs)

Draw lines, rectangles or polygons onto a colour or binary image. Can be used to connect. disconnect or erase contours.

Parameters
  • obj_input (array or container) – input object

  • overwrite (bool, optional) – if a container is supplied, or when working from the pype, should any exsting drawings be overwritten

  • tool ({line, polygon, rectangle} str, optional) – type of tool to use for drawing

  • line_colour ({"black", "white", "green", "red", "blue"} str, optional) – line or filling (for rectangle and polygon) colour

  • line_width (int, optional) – line width

Returns

image – image with drawings

Return type

array or container

phenopype.core.segmentation.find_contours(obj_input, df_image_data=None, approximation='simple', retrieval='ext', offset_coords=0, 0, min_nodes=3, max_nodes=inf, min_area=0, max_area=inf, min_diameter=0, max_diameter=inf, subset=None)

Find objects in binarized image. The input image needs to be a binarized image or a container on which a binarizing segmentation algorithm (e.g. threshold or watershed) has been performed. A flavor of different approximation algorithms and retrieval intstructions can be applied. The contours can also be filtered for minimum area, diameter or nodes (= number of corners).

Parameters
  • obj_input (array or container) – input object (binary)

  • df_image_data (DataFrame, optional) – an existing DataFrame containing image metadata

  • approximation ({"none", "simple", "L1", "KCOS"] str, optional) –

    contour approximation algorithm:
    • none: no approximation, all contour coordinates are returned

    • simple: only the minimum coordinates required are returned

    • L1: Teh-Chin chain approximation algorithm

    • KCOS: Teh-Chin chain approximation algorithm

  • retrieval ({"ext", "list", "tree", "ccomp", "flood"} str, optional) –

    contour retrieval procedure:
    • ext: retrieves only the extreme outer contours

    • list: retrieves all of the contours without establishing any hierarchical relationships

    • tree: retrieves all of the contours and reconstructs a full hierarchy of nested contours

    • ccomp: retrieves all of the contours and organizes them into a two-level hierarchy (parent and child)

    • flood: flood-fill algorithm

  • offset_coords (tuple, optional) – offset by which every contour point is shifted.

  • subset ({"parent", "child"} str, optional) – retain only a subset of the parent-child order structure

  • min_nodes (int, optional) – minimum number of coordinates

  • max_nodes (int, optional) – maximum number of coordinates

  • min_area (int, optional) – minimum contour area in pixels

  • max_area (int, optional) – maximum contour area in pixels

  • min_diameter (int, optional) – minimum diameter of boundary circle

  • max_diameter (int, optional) – maximum diameter of boundary circle

Returns

df_contours – contains contours

Return type

DataFrame or container

phenopype.core.segmentation.morphology(obj_input, kernel_size=5, shape='rect', operation='close', iterations=1)

Performs advanced morphological transformations using erosion and dilation as basic operations. Provides different kernel shapes and a suite of operation types (read more about morphological operations here: https://docs.opencv.org/master/db/df6/tutorial_erosion_dilatation.html)

Parameters
  • obj_input (array or container) – input object

  • kernel_size (int, optional) – size of the morphology kernel (has to be odd - even numbers will be ceiled)

  • shape ({"rect", "cross", "ellipse"} str, optional) – shape of the kernel

  • operation ({erode, dilate, open, close, gradient, tophat, blackhat, hitmiss} str, optional) –

    the morphology operation to be performed:
    • erode: remove pixels from the border

    • dilate: add pixels to the border

    • open: erosion followed by dilation

    • close: dilation followed by erosion

    • gradient: difference between dilation and erosion of an input image

    • tophat: difference between input image and opening of input image

    • blackhat: difference between input image and closing of input image

    • hitmiss: find patterns in binary images (read more here: https://docs.opencv.org/master/db/d06/tutorial_hitOrMiss.html)

  • iterations (int, optional) – number of times to run morphology operation

Returns

image – processed image

Return type

array or container

phenopype.core.segmentation.threshold(obj_input, df_masks=None, method='otsu', constant=1, blocksize=99, value=127, channel='gray', invert=False)

Applies a fixed-level threshold to create a binary image from a grayscale image or a multichannel image (gray, red, green or blue channel can be selected). Three types of thresholding algorithms are supported. Binary mask coordinates can be supplied to include or exclude areas.

Parameters
  • obj_input (array or container) – input object

  • df_masks (DataFrame, optional) – contains mask coordinates

  • method ({"otsu", "adaptive", "binary"} str, optional) –

    type of thresholding algorithm:
    • otsu: use Otsu algorithm to choose the optimal threshold value

    • adaptive: dynamic threshold values across image (uses arguments “blocksize” and “constant”)

    • binary: fixed threshold value (uses argument “value”)

  • blocksize (int, optional) – Size of a pixel neighborhood that is used to calculate a threshold value for the pixel (has to be odd - even numbers will be ceiled; for “adaptive” method)

  • constant (int, optional) – value to subtract from binarization output (for “adaptive” method)

  • value ({between 0 and 255} int, optional) – thesholding value (for “binary” method)

  • {"gray", "red", "green", "blue"} (channel) – which channel of the image to use for thresholding

  • invert (bool, optional) – invert all pixel values PRIOR to applying threshold values

Returns

image – binary image

Return type

array or container

phenopype.core.segmentation.watershed(obj_input, image_thresh=None, iterations=1, kernel_size=3, distance_cutoff=0.8, distance_mask=0, distance_type='l1')

Performs non-parametric marker-based segmentation - useful if many detected contours are touching or overlapping with each other. Input image should be a binary image that serves as the true background. Iteratively, edges are eroded, the difference serves as markers.

Parameters
  • obj_input (array or container) – input object

  • image_thresh (array, optional) – thresholded image n

  • kernel_size (int, optional) – size of the diff-kernel (has to be odd - even numbers will be ceiled)

  • iterations (int, optional) – number of times to apply diff-operation

  • distance_cutoff ({between 0 and 1} float, optional) – watershed distance transform cutoff (larger values discard more pixels)

  • distance_mask ({0, 3, 5} int, optional) – size of distance mask - not all sizes work with all distance types (will be coerced to 0)

  • distance_type ({"user", "l1", "l2", "C", "l12", "fair", "welsch", "huber"} str, optional) – distance transformation type

Returns

image – binary image

Return type

array or container

Measurement

phenopype.core.measurement.colour_intensity(obj_input, df_image_data=None, df_contours=None, channels='gray', background=False, background_ext=20)

Measures pixel values within the provided contours, either across all channels (“gray”) or for each channel separately (“rgb”). Measures mean and standard deviation :param obj_input: input object :type obj_input: array or container :param df_image_data: an existing DataFrame containing image metadata, will be added to

output DataFrame

Parameters
  • df_contours (DataFrame, optional) – contains the contours

  • channels ({"gray", "rgb"} str, optional) – for which channels should pixel intensity be measured

  • background (bool, optional) – measure the pixels of the background in an extended (background_ext) bounding box around the contour

  • background_ext (in, optional) – value in pixels by which the bounding box should be extended

Returns

df_colours – contains the pixel intensities

Return type

DataFrame or container

phenopype.core.measurement.landmarks(obj_input, df_image_data=None, overwrite=False, point_colour='green', point_size='auto', label_colour='black', label_size='auto', label_width='auto', **kwargs)

Place landmarks.

Parameters
  • obj_input (array or container) – input object

  • df_image_data (DataFrame, optional) – an existing DataFrame containing image metadata, will be added to landmark output DataFrame

  • overwrite (bool, optional) – if working using a container, or from a phenopype project directory, should existing landmarks be overwritten

  • point_colour ({"green", "red", "blue", "black", "white"} str, optional) – landmark point colour

  • point_size (int, optional) – landmark point size in pixels

  • label_colour ({"black", "white", "green", "red", "blue"} str, optional) – landmark label colour.

  • label_size (int, optional) – landmark label font size (scaled to image)

  • label_width (int, optional) – landmark label font width (scaled to image)

Returns

df_masks – contains landmark coordiantes

Return type

DataFrame or container

phenopype.core.measurement.polylines(obj_input, df_image_data=None, overwrite=False, line_width='auto', line_colour='blue', **kwargs)

Set points, draw a connected line between them, and measure its length.

Parameters
  • obj_input (array or container) – input object

  • df_image_data (DataFrame, optional) – an existing DataFrame containing image metadata, will be added to output DataFrame

  • overwrite (bool, optional) – if working using a container, or from a phenopype project directory, should existing polylines be overwritten

  • line_width (int, optional) – width of polyline

Returns

df_polylines – contains the drawn polylines

Return type

DataFrame or container

phenopype.core.measurement.skeletonize(obj_input, df_image_data=None, df_contours=None, thinning='zhangsuen', padding=10)

Applies a binary blob thinning operation, to achieve a skeletization of the input image using the technique, i.e. retrieve the topological skeleton (https://en.wikipedia.org/wiki/Topological_skeleton), using the algorithms of Thang-Suen or Guo-Hall.

Parameters
  • obj_input (array or container) – input object (binary)

  • df_image_data (DataFrame, optional) – an existing DataFrame containing image metadata, will be added to contour output DataFrame

  • df_contours (DataFrame, optional) – contains contours

  • thinning ({"zhangsuen", "guohall"} str, optional) – type of thinning algorithm to apply

Returns

image – thinned binary image

Return type

array or container

phenopype.core.measurement.shape_features(obj_input, df_contours=None, resize=True, resize_to=100, return_basic=True, return_moments=False, return_hu_moments=True)

Collects a set of 41 shape descriptors from every contour. There are three sets of descriptors: basic shape descriptors, moments, and hu moments. Two additional features, contour area and diameter are already provided by the find_contours function. https://docs.opencv.org/3.4.9/d3/dc0/group__imgproc__shape.html

Of the basic shape descriptors, all 10 are translational invariants, 8 are rotation invariant (rect_height and rect_width are not) and 4 are also scale invariant (circularity, compactness, roundness, solidity). https://en.wikipedia.org/wiki/Shape_factor_(image_analysis_and_microscopy)

The moments set encompasses 10 raw spatial moments (some are translation and rotation invariants, but not all), 7 central moments (all translational invariant) and 7 central normalized moments (all translational and scale invariant). https://en.wikipedia.org/wiki/Image_moment

The 7 hu moments are derived of the central moments, and are all translation, scale and rotation invariant. http://www.sci.utah.edu/~gerig/CS7960-S2010/handouts/Hu.pdf

Basic shape descriptors:

circularity = 4 * np.pi * contour_area / contour_perimeter_length^2 compactness = √(4 * contour_area / pi) / contour_diameter min_rect_max = minimum bounding rectangle major axis min_rect_min = minimum bounding rectangle minor axis perimeter_length = total length of contour perimenter rect_height = height of the bounding rectangle (“caliper dim 1”) rect_width = width of the bounding rectangle (“caliper dim 2”) roundness = (4 * contour_area) / pi * contour_perimeter_length^2 solidity = contour_area / convex_hull_area tri_area = area of minimum bounding triangle

Moments:

raw moments = m00, m10, m01, m20, m11, m02, m30, m21, m12, m03 central moments = mu20, mu11, mu02, mu30, mu21, mu12, mu03, normalized central moments = nu20, nu11, nu02, nu30, nu21, nu12, nu03

Hu moments:

hu moments = hu1, hu2, hu3, hu4, hu5, hu6, hu7

Parameters
  • obj_input (array or container) – input object

  • df_contours (DataFrame, optional) – contains the contours

  • return_basic (True, opational) – append the basic shape descriptors to a provided contour DataFrame

  • return_moments (False, optional) – append the basic shape descriptors to a provided contour DataFrame

  • return_hu_moments (False, optional) – append the basic shape descriptors to a provided contour DataFrame

Returns

df_contours – contains contours, and added features

Return type

DataFrame or container

phenopype.core.measurement.texture_features(obj_input, df_image_data=None, df_contours=None, channels='gray', background=False, background_ext=20, min_diameter=5)

Measures pixel values within the provided contours, either across all channels (“gray”) or for each channel separately (“rgb”). Measures mean and standard deviation

Parameters
  • obj_input (array or container) – input object

  • df_image_data (DataFrame, optional) – an existing DataFrame containing image metadata, will be added to output DataFrame

  • df_contours (DataFrame, optional) – contains the contours

  • channels ({"gray", "rgb"} str, optional) – for which channels should pixel intensity be measured

  • background (bool, optional) – measure the pixels of the background in an extended (background_ext) bounding box around the contour

  • background_ext (in, optional) – value in pixels by which the bounding box should be extended

  • min_diameter (int, optional) – minimum diameter of the contour

Returns

df_textures – contains the pixel intensities

Return type

DataFrame or container

Visualization

phenopype.core.visualization.select_canvas(obj_input, canvas='image_mod', multi=True)

Isolate a colour channel from an image or select canvas for the pype method.

Parameters
  • obj_input (array or container) – input object

  • canvas ({"mod", "bin", "gray", "raw", "red", "green", "blue"} str, optional) – the type of canvas to be used for visual feedback. some types require a function to be run first, e.g. “bin” needs a segmentation algorithm to be run first. black/white images don’t have colour channels. coerced to 3D array by default

  • multi (bool, optional) – coerce returned array to multilevel (3-level)

Returns

obj_input – canvas can be called with “obj_input.canvas”.

Return type

container

phenopype.core.visualization.draw_contours(obj_input, df_contours=None, offset_coords=None, label=True, fill=0.3, fill_colour=None, mark_holes=True, level=3, line_colour='green', label_size='auto', label_colour='black', line_width='auto', label_width='auto', skeleton=True, watershed=False, bounding_box=False, bounding_box_ext=20)

Draw contours and their labels onto a canvas. Can be filled or empty, offset coordinates can be supplied. This will also draw the skeleton, if the argument “skeleton=True” and the supplied “df_contour” contains a “skeleton_coords” column.

Parameters
  • obj_input (array or container) – input object

  • df_contours (DataFrame, optional) – contains the contours

  • offset_coords (tuple, optional) – offset coordinates, will be added to all contours

  • label (bool, optional) – draw contour label

  • fill (float, optional) – background transparency for contour fill (0=no fill).

  • fill_colour ({"green", "red", "blue", "black", "white"} str, optional) – contour fill colour - if not specified, defaults to line colour

  • mark_holes (bool, optional) – contours located inside other contours (i.e. their holes) will be highlighted in red

  • level (int, optional) – the default is 3.

  • line_colour ({"green", "red", "blue", "black", "white"} str, optional) – contour line colour

  • line_width (int, optional) – contour line width

  • label_colour ({"black", "white", "green", "red", "blue"} str, optional) – contour label colour.

  • label_size (int, optional) – contour label font size (scaled to image)

  • label_width (int, optional) – contour label font thickness

  • watershed (bool, optional) – indicates if a watershed-procedure has been performed. formats the coordinate colours accordingly (excludes “mark_holes option”)

  • bounding_box (bool, optional) – draw bounding box around the contour

  • bounding_box_ext (in, optional) – value in pixels by which the bounding box should be extended

Returns

image – image with contours

Return type

array or container

phenopype.core.visualization.draw_landmarks(obj_input, df_landmarks=None, point_colour='green', point_size='auto', label_colour='black', label_size='auto', label_width='auto')

Draw landmarks into an image.

Parameters
  • obj_input (array or container) – input object

  • df_landmarks (DataFrame, optional) – should contain contour coordinates as an array in a df cell

  • point_colour ({"green", "red", "blue", "black", "white"} str, optional) – landmark point colour

  • point_size (int, optional) – landmark point size in pixels

  • label_colour ({"black", "white", "green", "red", "blue"} str, optional) – landmark label colour.

  • label_size (int, optional) – landmark label font size (scaled to image)

  • label_width (int, optional) – landmark label font width (scaled to image)

Returns

image – image with landmarks

Return type

array or container

phenopype.core.visualization.draw_masks(obj_input, select=None, df_masks=None, line_colour='blue', line_width='auto', label=False, label_size='auto', label_colour='black', label_width='auto')

Draw masks into an image. This function is also used to draw the perimeter of a created or detected reference scale card.

Parameters
  • obj_input (array or container) – input object

  • select (str or list) – select a subset of masks to display

  • df_masks (DataFrame, optional) – contains mask coordinates and label

  • line_colour ({"blue", "red", "green", "black", "white"} str, optional) – mask line colour

  • line_width (int, optional) – mask line width

  • label_colour ({"black", "white", "green", "red", "blue"} str, optional) – mask label colour.

  • label_size (int, optional) – mask label font size (scaled to image)

  • label_width (int, optional) – mask label font width (scaled to image)

Returns

image – image with masks

Return type

array or container

phenopype.core.visualization.draw_polylines(obj_input, line_colour='blue', line_width='auto')

Draw polylines onto an image.

Parameters
  • obj_input (array or container) – input object

  • line_colour ({"blue", "red", "green", "black", "white"} str, optional) – polyline colour

  • line_width (int, optional) – polyline width

Returns

image – image with polylines

Return type

array or container

Export

phenopype.core.export.save_canvas(obj_input, overwrite=True, dirpath=None, save_suffix=None, name='', resize=0.5)

Save a canvas (processed image).

Parameters
  • obj_input (array or container) – input object

  • overwrite (bool, optional) – overwrite flag in case file exists

  • dirpath (str, optional) – folder to save file in

  • save_suffix (str, optional) – suffix to append to filename

  • name (str, optional) – custom name for file

  • resize (float, optional) – resize factor for the image (1 = 100%, 0.5 = 50%, 0.1 = 10% of original size).

phenopype.core.export.save_colours(obj_input, overwrite=True, dirpath=None, save_suffix=None, round_digits=1)

Save colour intensities to csv.

Parameters
  • obj_input (DataFrame or container) – input object

  • overwrite (bool optional) – overwrite csv if it already exists

  • dirpath (str, optional) – location to save df

  • round_digits (int, optional) – number of digits to round to

  • save_suffix (str, optional) – suffix to append to filename

phenopype.core.export.save_contours(obj_input, overwrite=True, dirpath=None, save_suffix=None, save_coords=False, convert_coords=True, subset=None)

Save contour coordinates and features to csv. This also saves skeletonization ouput if the data is contained in the provided DataFrame.

Parameters
  • obj_input (DataFrame or container) – input object

  • overwrite (bool optional) – overwrite csv if it already exists

  • dirpath (str, optional) – location to save df

  • save_suffix (str, optional) – suffix to append to filename

  • save_coords (bool, optional) – save the contour coordinates

  • convert_coords (bool, optional) – convert the coordinates from array to x and y column

  • subset ({"parent", "child"} str, optional) – save only a subset of the parent-child order structure

  • round_digits (int, optional) – number of digits to round to

phenopype.core.export.save_drawing(obj_input, overwrite=True, dirpath=None)

Save drawing coordinates to attributes.yaml

Parameters
  • obj_input (DataFrame or container) – input object

  • overwrite (bool optional) – overwrite if drawing already exists

  • dirpath (str, optional) – location to save df

phenopype.core.export.save_data_entry(obj_input, overwrite=True, dirpath=None)

Save data entry to attributes.yaml

Parameters
  • obj_input (DataFrame or container) – input object

  • overwrite (bool optional) – overwrite if entry already exists

  • dirpath (str, optional) – location to save df

phenopype.core.export.save_landmarks(obj_input, overwrite=True, dirpath=None, save_suffix=None)

Save landmark coordinates to csv.

Parameters
  • obj_input (DataFrame or container) – input object

  • overwrite (bool optional) – overwrite csv if it already exists

  • dirpath (str, optional) – location to save df

  • save_suffix (str, optional) – suffix to append to filename

phenopype.core.export.save_masks(obj_input, overwrite=True, dirpath=None, save_suffix=None)

Save mask coordinates and information (“include”” and “label”) to csv.

Parameters
  • obj_input (DataFrame or container) – input object

  • overwrite (bool optional) – overwrite csv if it already exists

  • dirpath (str, optional) – location to save df

  • save_suffix (str, optional) – suffix to append to filename

phenopype.core.export.save_polylines(obj_input, overwrite=True, dirpath=None, save_suffix=None)

Save polylines to csv.

Parameters
  • obj_input (DataFrame or container) – input object

  • overwrite (bool optional) – overwrite csv if it already exists

  • dirpath (str, optional) – location to save df

  • save_suffix (str, optional) – suffix to append to filename

phenopype.core.export.save_scale(obj_input, overwrite=True, dirpath=None)

Save a created or detected scale to attributes.yaml

Parameters
  • obj_input (DataFrame or container) – input object

  • overwrite (bool optional) – overwrite csv if it already exists

  • dirpath (str, optional) – location to save df

phenopype.core.export.save_textures(obj_input, overwrite=True, dirpath=None, save_suffix=None, round_digits=1, name='')

Save colour intensities to csv.

Parameters
  • obj_input (DataFrame or container) – input object

  • overwrite (bool optional) – overwrite csv if it already exists

  • dirpath (str, optional) – location to save df

  • round_digits (int, optional) – number of digits to round to

  • save_suffix (str, optional) – suffix to append to filename

phenopype.core.export.save_shapes(obj_input, overwrite=True, dirpath=None, save_suffix=None, round_digits=1)

Save colour intensities to csv.

Parameters
  • obj_input (DataFrame or container) – input object

  • overwrite (bool optional) – overwrite csv if it already exists

  • dirpath (str, optional) – location to save df

  • round_digits (int, optional) – number of digits to round to

  • save_suffix (str, optional) – suffix to append to filename

Video analysis

These classes provide basic motion tracking features via foreground-background subtraction - for more details see the OpenCV docs 2 and Tutorial 6.

Motion tracker

class phenopype.tracking.motion_tracker(video_path, at_frame=1)

Bases: object

Initialize motion tracker class; extract information (length, fps, codec, etc.) from input video and pass to other tracking methods.

Parameters
  • video_path (str) – path to video

  • at_frame (int, optional) – frame index to be used to extract the video information

video_output(video_format=None, save_suffix='out', dirpath=None, fps=None, save_colour=None, dimensions=None, resize=1)

Set properties of output video file. Most settings can be left blank, so that settings from the input video will be applied

Parameters
  • video_format (str, optional) – format of the output video file. needs to be a fourcc-string https://www.fourcc.org/codecs.php

  • save_suffix (str, optional) – name for the output video file (defaut: “input-filename” + _out.avi)

  • dirpath (str, optional) – save directory for the output video file. will be created if not existing

  • fps (int, optional) – frames per second of the output video file

  • dimensions (tuple, optional) – dimensions (width, length) of the output video file

  • resize (int, optional) – factor by which to resize the output video dimensions

  • save_colour (bool, optional) – should the saved video frames be in colour

detection_settings(skip=5, warmup=0, start_after=0, finish_after=0, history=60, threshold=10, detect_shadows=True, mode='MOG', methods=None, c_mask=False, c_mask_shape='rect', c_mask_size=50)

Set properties of output video file. Most settings can be left at their default value.

Parameters
  • skip (int, optional) – how many frames to skip between each capture

  • warmup (int, optional) – warmup period in seconds for the background subtractor

  • start_after (int, optional) – start after X seconds

  • finish_after (int, optional) – finish after X seconds

  • history (int, optional) – how many frames to use for fg-bg subtraction algorithm

  • threshold (int, optional) – sensitivity-level for fg-bg subtraction algorithm (lower = more sensitive)

  • detect_shadows (bool, optional) – attempt to detect shadows - will be returned as gray pixels

  • mode ({"MOG", "KNN"} str, optional) – type of fg-bg subtraction algorithm

  • methods (method or list of methods, optional) – list with tracking_method objects

  • c_mask (bool, optional) – consecutive masking. if multiple methods are defined, the objects detected first will mask the objects detected in subsequent methods

  • c_mask_shape ({"rect", "ellipse", "contour"} str, optional) – which shape should the consecutive mask have

  • c_mask_size (int, optional) – area in pixels that is added around the mask

run_tracking(feedback=True, canvas='overlay', overlay_weight=0.5)

Start motion tracking procedure. Enable or disable video feedback, output and select canvas (overlay of detected objects, foreground mask, input video).

Parameters
  • feedback (bool, optional) – show output of tracking

  • canvas ({"overlay","fgmask","input"} str, optional) – the background for the ouput video

  • overlay_weight (float (default: 0.5)) – if canvas=”overlay”, how transparent should the overlay should be

Tracking methods

class phenopype.tracking.tracking_method(label='m1', blur=5, threshold=127, remove_shadows=True, min_length=0, max_length=inf, min_area=0, max_area=inf, mode='multiple', overlay_colour='red', operations=[])

Bases: object

Constructs a tracking method that can be supplied to the motion_tracker class.

Parameters
  • label (str, optional) – label for all objects detected by this method

  • blur (int, optional) – blurring of fgbg-mask (kernel size)

  • threshold (int, optional) – binarization of fgbg-mask after blurring (threshold value)

  • remove_shadows (bool, optional) – if motion_tracker has detect_shadows=True, they can be removed here

  • min_area (int, optional) – minimum contour area in pixels to be included

  • max_area (int, optional) – maximum contour area in pixels to be included

  • min_length (int, optional) – minimum diameter of boundary circle to be included

  • max_length (int, optional) – maximum diameter of boundary circle to be included

  • mode ({"single", "multiple"} str, optional) – track “multiple”, or “single” (biggest by diameter) objects

  • remove_shadows – remove shadows if shadow-detection is actived in MOG-algorithm

  • overlay_colour ({"red", "green", "blue", "black", "white"} str, optional) – which colour should tracked objects have

  • operations (list (default: ["length", "area"])) –

    determines the type of operations to be performed on the detected objects:
    • ”diameter” of the bounding circle of our object

    • ”area” within the contour of our object

    • ”grayscale” mean and standard deviation of grayscale pixel values inside the object contours

    • ”grayscale_background” background within boundingbox of contour

    • ”bgr” mean and standard deviation of blue, green and red pixel values inside the object contours