Example 4: Video analysis - predator prey interactions

This example features analysis of predator-prey interactions of one threespine stickleback (Gasterosteus aculeatus) and 20 benthic isopods (Asellus aquaticus). The goal is to extract movement trajectories of both species in response to each other, to see if isopods are reacting to the fish, and the other way round. At the same time, isopod pigmentation is measured in every frame to measure whether the fish has preferences for a particular phenotype (dark vs. light pigmentation, large bodied individuals).

The full video can be found here: https://vimeo.com/283075068

Fish movement - Fish movement is extracted by following the movement of the single biggest object.

Isopod movement - Movement of isopods over 15 seconds.

Video tracking with phenopype

import os
import pandas as pd
import phenopype as pp
import trackpy as tp # can be installed using pip

video_path =  r"images/isopods_fish.mp4"
out_dir =  r"../_temp/output/ex4"

First we initialize the motion tracker with our example video, and specify the video output.

mt = pp.motion_tracker(video_path)

mt.video_output(save_suffix="v1", dirpath=out_dir)

--------------------------------------------------------------
Input video properties - "isopods_fish.mp4":

Frames per second: 29.97000099996667
N frames: 450
Length: 00:15 (mm:ss)
Dimensions: (960, 540)
Colour video: True
FourCC code: avc1
--------------------------------------------------------------
Save folder ../_temp/output/ex4 does not exist - create?.y


--------------------------------------------------------------
Output video settings - "isopods_fish.mp4":

Save name: isopods_fish_v1.mp4
Save dir: E:\git_repos\phenopype\_temp\output\ex4
Frames per second: 29.97000099996667
Dimensions: (960, 540)
Colour video: True
Format (FourCC code): avc1
--------------------------------------------------------------

Then we create a mask to only include the gravel area - this will exclude the walls where reflections and floating particles may result in false positives. A second mask in the center of the area can later be used to compare whether isopods tend to stay more on the side (shy behavior) or whether they move freely (bold behavior). In each captured frame there will be an entry for each isopods in which area it was detected.

Masking - on one hand we want to exclude unwanted areas, on the other hand, we want to see when isopds stays in the center or not.

pp.preprocessing.create_mask(mt, label="full arena")
pp.preprocessing.create_mask(mt, label="center")

- create mask

Now we set up two tracking_methods: the “fish” method is set to "single" and will only capture the largest object, whereas the isopod method will capture all objects smaller than 30 pixels. You can play around with blur and threshold settings to see if you can get better results. The operations argument is used to return specific information of the detected objects. We specify a few parameters to retrieve phenotypic information of the isopods. Later, we can then analyze which phenotypes were foraged on which sediment background.

fish = pp.tracking_method(label="fish", remove_shadows=True, min_length=30,
                          overlay_colour="red", mode="single",
                          blur=15, # bigger blurring kernel
                          threshold=200 #higher sensitivity
                         )
isopod = pp.tracking_method(label="isopod", remove_shadows=True, max_length=30,
                            overlay_colour="green", mode="multiple",
                            blur=9, # smaller blurring kernel
                            threshold=180, # lower sensitivity
                            operations=["diameter",  # isopod size
                                        "area",      # isopod area
                                        "grayscale", # isopod pigmentation
                                        "grayscale_background"] # background darkness
                           )

Finally, we pass the methods on the detection settings, and also activate consecutive masking (c_mask=True). This option will prohibit repeated detection of objects when several tracking methods are applied. For example, inside the detected fish-area sometimes isopod objects are detected, due to artifacts or incomplete subtraction results. Consecutive masking will “block” an area after the first method has been applied - the order by which methods are applied matters. The following settings will create a rectangle shaped mask around the fish with a 200 pixel border.

mt.detection_settings(methods=[fish, isopod],
                     c_mask=True,
                     c_mask_shape="rect",
                     c_mask_size=200)

{'blur_kernel': 15,
 'label': 'fish',
 'max_area': inf,
 'max_length': inf,
 'min_area': 0,
 'min_length': 30,
 'mode': 'single',
 'operations': [],
 'overlay_colour': (0, 0,
                    255),
 'remove_shadows': True,
 'threshold_value': 200}
{'blur_kernel': 9,
 'label': 'isopod',
 'max_area': inf,
 'max_length': 30,
 'min_area': 0,
 'min_length': 0,
 'mode': 'multiple',
 'operations': ['diameter',
                'area',
                'grayscale',
                'grayscale_background'],
 'overlay_colour': (0, 255,
                    0),
 'remove_shadows': True,
 'threshold_value': 180}


--------------------------------------------------------------
Motion detection settings - "isopods_fish.mp4":

Background-subtractor: MOG
History: 60 seconds
Sensitivity: 10
Read every nth frame: 5
Detect shadows: True
Start after n seconds: 0
Finish after n seconds:  -
--------------------------------------------------------------

coordinates = mt.run_tracking()

Time: 00:00/00:15 - Frames: 1/450
Time: 00:00/00:15 - Frames: 2/450
Time: 00:00/00:15 - Frames: 3/450
Time: 00:00/00:15 - Frames: 4/450
Time: 00:00/00:15 - Frames: 5/450 - captured
Time: 00:00/00:15 - Frames: 6/450
Time: 00:00/00:15 - Frames: 7/450
Time: 00:00/00:15 - Frames: 8/450
Time: 00:00/00:15 - Frames: 9/450
Time: 00:00/00:15 - Frames: 10/450 - captured
Time: 00:00/00:15 - Frames: 11/450
Time: 00:00/00:15 - Frames: 12/450
Time: 00:00/00:15 - Frames: 13/450
Time: 00:00/00:15 - Frames: 14/450
Time: 00:00/00:15 - Frames: 15/450 - captured
Time: 00:00/00:15 - Frames: 16/450
Time: 00:00/00:15 - Frames: 17/450
Time: 00:00/00:15 - Frames: 18/450
Time: 00:00/00:15 - Frames: 19/450
Time: 00:00/00:15 - Frames: 20/450 - captured
Time: 00:00/00:15 - Frames: 21/450
Time: 00:00/00:15 - Frames: 22/450
Time: 00:00/00:15 - Frames: 23/450
Time: 00:00/00:15 - Frames: 24/450
Time: 00:00/00:15 - Frames: 25/450 - captured
Time: 00:00/00:15 - Frames: 26/450
Time: 00:00/00:15 - Frames: 27/450
Time: 00:00/00:15 - Frames: 28/450
Time: 00:00/00:15 - Frames: 29/450
Time: 00:01/00:15 - Frames: 30/450 - captured
Time: 00:01/00:15 - Frames: 31/450
Time: 00:01/00:15 - Frames: 32/450
Time: 00:01/00:15 - Frames: 33/450
Time: 00:01/00:15 - Frames: 34/450
Time: 00:01/00:15 - Frames: 35/450 - captured
Time: 00:01/00:15 - Frames: 36/450
Time: 00:01/00:15 - Frames: 37/450
Time: 00:01/00:15 - Frames: 38/450
Time: 00:01/00:15 - Frames: 39/450
Time: 00:01/00:15 - Frames: 40/450 - captured
Time: 00:01/00:15 - Frames: 41/450
Time: 00:01/00:15 - Frames: 42/450
Time: 00:01/00:15 - Frames: 43/450
Time: 00:01/00:15 - Frames: 44/450
Time: 00:01/00:15 - Frames: 45/450 - captured
Time: 00:01/00:15 - Frames: 46/450
Time: 00:01/00:15 - Frames: 47/450
Time: 00:01/00:15 - Frames: 48/450
Time: 00:01/00:15 - Frames: 49/450
Time: 00:01/00:15 - Frames: 50/450 - captured
Time: 00:01/00:15 - Frames: 51/450
Time: 00:01/00:15 - Frames: 52/450
Time: 00:01/00:15 - Frames: 53/450
Time: 00:01/00:15 - Frames: 54/450
Time: 00:01/00:15 - Frames: 55/450 - captured
Time: 00:01/00:15 - Frames: 56/450
Time: 00:01/00:15 - Frames: 57/450
Time: 00:01/00:15 - Frames: 58/450
Time: 00:01/00:15 - Frames: 59/450
Time: 00:02/00:15 - Frames: 60/450 - captured
Time: 00:02/00:15 - Frames: 61/450
Time: 00:02/00:15 - Frames: 62/450
Time: 00:02/00:15 - Frames: 63/450
Time: 00:02/00:15 - Frames: 64/450
Time: 00:02/00:15 - Frames: 65/450 - captured
Time: 00:02/00:15 - Frames: 66/450
Time: 00:02/00:15 - Frames: 67/450
Time: 00:02/00:15 - Frames: 68/450
Time: 00:02/00:15 - Frames: 69/450
Time: 00:02/00:15 - Frames: 70/450 - captured
Time: 00:02/00:15 - Frames: 71/450
Time: 00:02/00:15 - Frames: 72/450
Time: 00:02/00:15 - Frames: 73/450
Time: 00:02/00:15 - Frames: 74/450
Time: 00:02/00:15 - Frames: 75/450 - captured
Time: 00:02/00:15 - Frames: 76/450
Time: 00:02/00:15 - Frames: 77/450
Time: 00:02/00:15 - Frames: 78/450
Time: 00:02/00:15 - Frames: 79/450
Time: 00:02/00:15 - Frames: 80/450 - captured
Time: 00:02/00:15 - Frames: 81/450
Time: 00:02/00:15 - Frames: 82/450
Time: 00:02/00:15 - Frames: 83/450
Time: 00:02/00:15 - Frames: 84/450
Time: 00:02/00:15 - Frames: 85/450 - captured
Time: 00:02/00:15 - Frames: 86/450
Time: 00:02/00:15 - Frames: 87/450
Time: 00:02/00:15 - Frames: 88/450
Time: 00:02/00:15 - Frames: 89/450
Time: 00:03/00:15 - Frames: 90/450 - captured
Time: 00:03/00:15 - Frames: 91/450
Time: 00:03/00:15 - Frames: 92/450
Time: 00:03/00:15 - Frames: 93/450
Time: 00:03/00:15 - Frames: 94/450
Time: 00:03/00:15 - Frames: 95/450 - captured
Time: 00:03/00:15 - Frames: 96/450
Time: 00:03/00:15 - Frames: 97/450
Time: 00:03/00:15 - Frames: 98/450
Time: 00:03/00:15 - Frames: 99/450
Time: 00:03/00:15 - Frames: 100/450 - captured
Time: 00:03/00:15 - Frames: 101/450
Time: 00:03/00:15 - Frames: 102/450
Time: 00:03/00:15 - Frames: 103/450
Time: 00:03/00:15 - Frames: 104/450
Time: 00:03/00:15 - Frames: 105/450 - captured
Time: 00:03/00:15 - Frames: 106/450
Time: 00:03/00:15 - Frames: 107/450
Time: 00:03/00:15 - Frames: 108/450
Time: 00:03/00:15 - Frames: 109/450
Time: 00:03/00:15 - Frames: 110/450 - captured
Time: 00:03/00:15 - Frames: 111/450
Time: 00:03/00:15 - Frames: 112/450
Time: 00:03/00:15 - Frames: 113/450
Time: 00:03/00:15 - Frames: 114/450
Time: 00:03/00:15 - Frames: 115/450 - captured
Time: 00:03/00:15 - Frames: 116/450
Time: 00:03/00:15 - Frames: 117/450
Time: 00:03/00:15 - Frames: 118/450
Time: 00:03/00:15 - Frames: 119/450
Time: 00:04/00:15 - Frames: 120/450 - captured
Time: 00:04/00:15 - Frames: 121/450
Time: 00:04/00:15 - Frames: 122/450
Time: 00:04/00:15 - Frames: 123/450
Time: 00:04/00:15 - Frames: 124/450
Time: 00:04/00:15 - Frames: 125/450 - captured
Time: 00:04/00:15 - Frames: 126/450
Time: 00:04/00:15 - Frames: 127/450
Time: 00:04/00:15 - Frames: 128/450
Time: 00:04/00:15 - Frames: 129/450
Time: 00:04/00:15 - Frames: 130/450 - captured
Time: 00:04/00:15 - Frames: 131/450
Time: 00:04/00:15 - Frames: 132/450
Time: 00:04/00:15 - Frames: 133/450
Time: 00:04/00:15 - Frames: 134/450
Time: 00:04/00:15 - Frames: 135/450 - captured
Time: 00:04/00:15 - Frames: 136/450
Time: 00:04/00:15 - Frames: 137/450
Time: 00:04/00:15 - Frames: 138/450
Time: 00:04/00:15 - Frames: 139/450
Time: 00:04/00:15 - Frames: 140/450 - captured
Time: 00:04/00:15 - Frames: 141/450
Time: 00:04/00:15 - Frames: 142/450
Time: 00:04/00:15 - Frames: 143/450
Time: 00:04/00:15 - Frames: 144/450
Time: 00:04/00:15 - Frames: 145/450 - captured
Time: 00:04/00:15 - Frames: 146/450
Time: 00:04/00:15 - Frames: 147/450
Time: 00:04/00:15 - Frames: 148/450
Time: 00:04/00:15 - Frames: 149/450
Time: 00:05/00:15 - Frames: 150/450 - captured
Time: 00:05/00:15 - Frames: 151/450
Time: 00:05/00:15 - Frames: 152/450
Time: 00:05/00:15 - Frames: 153/450
Time: 00:05/00:15 - Frames: 154/450
Time: 00:05/00:15 - Frames: 155/450 - captured
Time: 00:05/00:15 - Frames: 156/450
Time: 00:05/00:15 - Frames: 157/450
Time: 00:05/00:15 - Frames: 158/450
Time: 00:05/00:15 - Frames: 159/450
Time: 00:05/00:15 - Frames: 160/450 - captured
Time: 00:05/00:15 - Frames: 161/450
Time: 00:05/00:15 - Frames: 162/450
Time: 00:05/00:15 - Frames: 163/450
Time: 00:05/00:15 - Frames: 164/450
Time: 00:05/00:15 - Frames: 165/450 - captured
Time: 00:05/00:15 - Frames: 166/450
Time: 00:05/00:15 - Frames: 167/450
Time: 00:05/00:15 - Frames: 168/450
Time: 00:05/00:15 - Frames: 169/450
Time: 00:05/00:15 - Frames: 170/450 - captured
Time: 00:05/00:15 - Frames: 171/450
Time: 00:05/00:15 - Frames: 172/450
Time: 00:05/00:15 - Frames: 173/450
Time: 00:05/00:15 - Frames: 174/450
Time: 00:05/00:15 - Frames: 175/450 - captured
Time: 00:05/00:15 - Frames: 176/450
Time: 00:05/00:15 - Frames: 177/450
Time: 00:05/00:15 - Frames: 178/450
Time: 00:05/00:15 - Frames: 179/450
Time: 00:06/00:15 - Frames: 180/450 - captured
Time: 00:06/00:15 - Frames: 181/450
Time: 00:06/00:15 - Frames: 182/450
Time: 00:06/00:15 - Frames: 183/450
Time: 00:06/00:15 - Frames: 184/450
Time: 00:06/00:15 - Frames: 185/450 - captured
Time: 00:06/00:15 - Frames: 186/450
Time: 00:06/00:15 - Frames: 187/450
Time: 00:06/00:15 - Frames: 188/450
Time: 00:06/00:15 - Frames: 189/450
Time: 00:06/00:15 - Frames: 190/450 - captured
Time: 00:06/00:15 - Frames: 191/450
Time: 00:06/00:15 - Frames: 192/450
Time: 00:06/00:15 - Frames: 193/450
Time: 00:06/00:15 - Frames: 194/450
Time: 00:06/00:15 - Frames: 195/450 - captured
Time: 00:06/00:15 - Frames: 196/450
Time: 00:06/00:15 - Frames: 197/450
Time: 00:06/00:15 - Frames: 198/450
Time: 00:06/00:15 - Frames: 199/450
Time: 00:06/00:15 - Frames: 200/450 - captured
Time: 00:06/00:15 - Frames: 201/450
Time: 00:06/00:15 - Frames: 202/450
Time: 00:06/00:15 - Frames: 203/450
Time: 00:06/00:15 - Frames: 204/450
Time: 00:06/00:15 - Frames: 205/450 - captured
Time: 00:06/00:15 - Frames: 206/450
Time: 00:06/00:15 - Frames: 207/450
Time: 00:06/00:15 - Frames: 208/450
Time: 00:06/00:15 - Frames: 209/450
Time: 00:07/00:15 - Frames: 210/450 - captured
Time: 00:07/00:15 - Frames: 211/450
Time: 00:07/00:15 - Frames: 212/450
Time: 00:07/00:15 - Frames: 213/450
Time: 00:07/00:15 - Frames: 214/450
Time: 00:07/00:15 - Frames: 215/450 - captured
Time: 00:07/00:15 - Frames: 216/450
Time: 00:07/00:15 - Frames: 217/450
Time: 00:07/00:15 - Frames: 218/450
Time: 00:07/00:15 - Frames: 219/450
Time: 00:07/00:15 - Frames: 220/450 - captured
Time: 00:07/00:15 - Frames: 221/450
Time: 00:07/00:15 - Frames: 222/450
Time: 00:07/00:15 - Frames: 223/450
Time: 00:07/00:15 - Frames: 224/450
Time: 00:07/00:15 - Frames: 225/450 - captured
Time: 00:07/00:15 - Frames: 226/450
Time: 00:07/00:15 - Frames: 227/450
Time: 00:07/00:15 - Frames: 228/450
Time: 00:07/00:15 - Frames: 229/450
Time: 00:07/00:15 - Frames: 230/450 - captured
Time: 00:07/00:15 - Frames: 231/450
Time: 00:07/00:15 - Frames: 232/450
Time: 00:07/00:15 - Frames: 233/450
Time: 00:07/00:15 - Frames: 234/450
Time: 00:07/00:15 - Frames: 235/450 - captured
Time: 00:07/00:15 - Frames: 236/450
Time: 00:07/00:15 - Frames: 237/450
Time: 00:07/00:15 - Frames: 238/450
Time: 00:07/00:15 - Frames: 239/450
Time: 00:08/00:15 - Frames: 240/450 - captured
Time: 00:08/00:15 - Frames: 241/450
Time: 00:08/00:15 - Frames: 242/450
Time: 00:08/00:15 - Frames: 243/450
Time: 00:08/00:15 - Frames: 244/450
Time: 00:08/00:15 - Frames: 245/450 - captured
Time: 00:08/00:15 - Frames: 246/450
Time: 00:08/00:15 - Frames: 247/450
Time: 00:08/00:15 - Frames: 248/450
Time: 00:08/00:15 - Frames: 249/450
Time: 00:08/00:15 - Frames: 250/450 - captured
Time: 00:08/00:15 - Frames: 251/450
Time: 00:08/00:15 - Frames: 252/450
Time: 00:08/00:15 - Frames: 253/450
Time: 00:08/00:15 - Frames: 254/450
Time: 00:08/00:15 - Frames: 255/450 - captured
Time: 00:08/00:15 - Frames: 256/450
Time: 00:08/00:15 - Frames: 257/450
Time: 00:08/00:15 - Frames: 258/450
Time: 00:08/00:15 - Frames: 259/450
Time: 00:08/00:15 - Frames: 260/450 - captured
Time: 00:08/00:15 - Frames: 261/450
Time: 00:08/00:15 - Frames: 262/450
Time: 00:08/00:15 - Frames: 263/450
Time: 00:08/00:15 - Frames: 264/450
Time: 00:08/00:15 - Frames: 265/450 - captured
Time: 00:08/00:15 - Frames: 266/450
Time: 00:08/00:15 - Frames: 267/450
Time: 00:08/00:15 - Frames: 268/450
Time: 00:08/00:15 - Frames: 269/450
Time: 00:09/00:15 - Frames: 270/450 - captured
Time: 00:09/00:15 - Frames: 271/450
Time: 00:09/00:15 - Frames: 272/450
Time: 00:09/00:15 - Frames: 273/450
Time: 00:09/00:15 - Frames: 274/450
Time: 00:09/00:15 - Frames: 275/450 - captured
Time: 00:09/00:15 - Frames: 276/450
Time: 00:09/00:15 - Frames: 277/450
Time: 00:09/00:15 - Frames: 278/450
Time: 00:09/00:15 - Frames: 279/450
Time: 00:09/00:15 - Frames: 280/450 - captured
Time: 00:09/00:15 - Frames: 281/450
Time: 00:09/00:15 - Frames: 282/450
Time: 00:09/00:15 - Frames: 283/450
Time: 00:09/00:15 - Frames: 284/450
Time: 00:09/00:15 - Frames: 285/450 - captured
Time: 00:09/00:15 - Frames: 286/450
Time: 00:09/00:15 - Frames: 287/450
Time: 00:09/00:15 - Frames: 288/450
Time: 00:09/00:15 - Frames: 289/450
Time: 00:09/00:15 - Frames: 290/450 - captured
Time: 00:09/00:15 - Frames: 291/450
Time: 00:09/00:15 - Frames: 292/450
Time: 00:09/00:15 - Frames: 293/450
Time: 00:09/00:15 - Frames: 294/450
Time: 00:09/00:15 - Frames: 295/450 - captured
Time: 00:09/00:15 - Frames: 296/450
Time: 00:09/00:15 - Frames: 297/450
Time: 00:09/00:15 - Frames: 298/450
Time: 00:09/00:15 - Frames: 299/450
Time: 00:10/00:15 - Frames: 300/450 - captured
Time: 00:10/00:15 - Frames: 301/450
Time: 00:10/00:15 - Frames: 302/450
Time: 00:10/00:15 - Frames: 303/450
Time: 00:10/00:15 - Frames: 304/450
Time: 00:10/00:15 - Frames: 305/450 - captured
Time: 00:10/00:15 - Frames: 306/450
Time: 00:10/00:15 - Frames: 307/450
Time: 00:10/00:15 - Frames: 308/450
Time: 00:10/00:15 - Frames: 309/450
Time: 00:10/00:15 - Frames: 310/450 - captured
Time: 00:10/00:15 - Frames: 311/450
Time: 00:10/00:15 - Frames: 312/450
Time: 00:10/00:15 - Frames: 313/450
Time: 00:10/00:15 - Frames: 314/450
Time: 00:10/00:15 - Frames: 315/450 - captured
Time: 00:10/00:15 - Frames: 316/450
Time: 00:10/00:15 - Frames: 317/450
Time: 00:10/00:15 - Frames: 318/450
Time: 00:10/00:15 - Frames: 319/450
Time: 00:10/00:15 - Frames: 320/450 - captured
Time: 00:10/00:15 - Frames: 321/450
Time: 00:10/00:15 - Frames: 322/450
Time: 00:10/00:15 - Frames: 323/450
Time: 00:10/00:15 - Frames: 324/450
Time: 00:10/00:15 - Frames: 325/450 - captured
Time: 00:10/00:15 - Frames: 326/450
Time: 00:10/00:15 - Frames: 327/450
Time: 00:10/00:15 - Frames: 328/450
Time: 00:10/00:15 - Frames: 329/450
Time: 00:11/00:15 - Frames: 330/450 - captured
Time: 00:11/00:15 - Frames: 331/450
Time: 00:11/00:15 - Frames: 332/450
Time: 00:11/00:15 - Frames: 333/450
Time: 00:11/00:15 - Frames: 334/450
Time: 00:11/00:15 - Frames: 335/450 - captured
Time: 00:11/00:15 - Frames: 336/450
Time: 00:11/00:15 - Frames: 337/450
Time: 00:11/00:15 - Frames: 338/450
Time: 00:11/00:15 - Frames: 339/450
Time: 00:11/00:15 - Frames: 340/450 - captured
Time: 00:11/00:15 - Frames: 341/450
Time: 00:11/00:15 - Frames: 342/450
Time: 00:11/00:15 - Frames: 343/450
Time: 00:11/00:15 - Frames: 344/450
Time: 00:11/00:15 - Frames: 345/450 - captured
Time: 00:11/00:15 - Frames: 346/450
Time: 00:11/00:15 - Frames: 347/450
Time: 00:11/00:15 - Frames: 348/450
Time: 00:11/00:15 - Frames: 349/450
Time: 00:11/00:15 - Frames: 350/450 - captured
Time: 00:11/00:15 - Frames: 351/450
Time: 00:11/00:15 - Frames: 352/450
Time: 00:11/00:15 - Frames: 353/450
Time: 00:11/00:15 - Frames: 354/450
Time: 00:11/00:15 - Frames: 355/450 - captured
Time: 00:11/00:15 - Frames: 356/450
Time: 00:11/00:15 - Frames: 357/450
Time: 00:11/00:15 - Frames: 358/450
Time: 00:11/00:15 - Frames: 359/450
Time: 00:12/00:15 - Frames: 360/450 - captured
Time: 00:12/00:15 - Frames: 361/450
Time: 00:12/00:15 - Frames: 362/450
Time: 00:12/00:15 - Frames: 363/450
Time: 00:12/00:15 - Frames: 364/450
Time: 00:12/00:15 - Frames: 365/450 - captured
Time: 00:12/00:15 - Frames: 366/450
Time: 00:12/00:15 - Frames: 367/450
Time: 00:12/00:15 - Frames: 368/450
Time: 00:12/00:15 - Frames: 369/450
Time: 00:12/00:15 - Frames: 370/450 - captured
Time: 00:12/00:15 - Frames: 371/450
Time: 00:12/00:15 - Frames: 372/450
Time: 00:12/00:15 - Frames: 373/450
Time: 00:12/00:15 - Frames: 374/450
Time: 00:12/00:15 - Frames: 375/450 - captured
Time: 00:12/00:15 - Frames: 376/450
Time: 00:12/00:15 - Frames: 377/450
Time: 00:12/00:15 - Frames: 378/450
Time: 00:12/00:15 - Frames: 379/450
Time: 00:12/00:15 - Frames: 380/450 - captured
Time: 00:12/00:15 - Frames: 381/450
Time: 00:12/00:15 - Frames: 382/450
Time: 00:12/00:15 - Frames: 383/450
Time: 00:12/00:15 - Frames: 384/450
Time: 00:12/00:15 - Frames: 385/450 - captured
Time: 00:12/00:15 - Frames: 386/450
Time: 00:12/00:15 - Frames: 387/450
Time: 00:12/00:15 - Frames: 388/450
Time: 00:12/00:15 - Frames: 389/450
Time: 00:13/00:15 - Frames: 390/450 - captured
Time: 00:13/00:15 - Frames: 391/450
Time: 00:13/00:15 - Frames: 392/450
Time: 00:13/00:15 - Frames: 393/450
Time: 00:13/00:15 - Frames: 394/450
Time: 00:13/00:15 - Frames: 395/450 - captured
Time: 00:13/00:15 - Frames: 396/450
Time: 00:13/00:15 - Frames: 397/450
Time: 00:13/00:15 - Frames: 398/450
Time: 00:13/00:15 - Frames: 399/450
Time: 00:13/00:15 - Frames: 400/450 - captured
Time: 00:13/00:15 - Frames: 401/450
Time: 00:13/00:15 - Frames: 402/450
Time: 00:13/00:15 - Frames: 403/450
Time: 00:13/00:15 - Frames: 404/450
Time: 00:13/00:15 - Frames: 405/450 - captured
Time: 00:13/00:15 - Frames: 406/450
Time: 00:13/00:15 - Frames: 407/450
Time: 00:13/00:15 - Frames: 408/450
Time: 00:13/00:15 - Frames: 409/450
Time: 00:13/00:15 - Frames: 410/450 - captured
Time: 00:13/00:15 - Frames: 411/450
Time: 00:13/00:15 - Frames: 412/450
Time: 00:13/00:15 - Frames: 413/450
Time: 00:13/00:15 - Frames: 414/450
Time: 00:13/00:15 - Frames: 415/450 - captured
Time: 00:13/00:15 - Frames: 416/450
Time: 00:13/00:15 - Frames: 417/450
Time: 00:13/00:15 - Frames: 418/450
Time: 00:13/00:15 - Frames: 419/450
Time: 00:14/00:15 - Frames: 420/450 - captured
Time: 00:14/00:15 - Frames: 421/450
Time: 00:14/00:15 - Frames: 422/450
Time: 00:14/00:15 - Frames: 423/450
Time: 00:14/00:15 - Frames: 424/450
Time: 00:14/00:15 - Frames: 425/450 - captured
Time: 00:14/00:15 - Frames: 426/450
Time: 00:14/00:15 - Frames: 427/450
Time: 00:14/00:15 - Frames: 428/450
Time: 00:14/00:15 - Frames: 429/450
Time: 00:14/00:15 - Frames: 430/450 - captured
Time: 00:14/00:15 - Frames: 431/450
Time: 00:14/00:15 - Frames: 432/450
Time: 00:14/00:15 - Frames: 433/450
Time: 00:14/00:15 - Frames: 434/450
Time: 00:14/00:15 - Frames: 435/450 - captured
Time: 00:14/00:15 - Frames: 436/450
Time: 00:14/00:15 - Frames: 437/450
Time: 00:14/00:15 - Frames: 438/450
Time: 00:14/00:15 - Frames: 439/450
Time: 00:14/00:15 - Frames: 440/450 - captured
Time: 00:14/00:15 - Frames: 441/450
Time: 00:14/00:15 - Frames: 442/450
Time: 00:14/00:15 - Frames: 443/450
Time: 00:14/00:15 - Frames: 444/450
Time: 00:14/00:15 - Frames: 445/450 - captured
Time: 00:14/00:15 - Frames: 446/450
Time: 00:14/00:15 - Frames: 447/450
Time: 00:14/00:15 - Frames: 448/450

After completing the tracking, we end up with a big data frame of all contours

coordinates.to_csv(os.path.join(out_dir, mt.name + "coordinates.csv"), sep=',')
coordinates

	frame_abs	frame	mins	secs	x	y	diameter	area	grayscale	grayscale_background	label	full arena	center
0	30	6	00	01	608.0	345.0	7.0	23.0	136.0	130.0	isopod	True	True
1	30	6	00	01	546.0	249.0	11.0	54.0	116.0	106.0	isopod	NaN	NaN
2	35	7	00	01	575.0	306.0	8.0	47.0	125.0	117.0	isopod	True	True
3	45	9	00	01	193.0	305.0	5.0	13.0	185.0	173.0	isopod	True	False
4	50	10	00	01	193.0	308.0	7.0	27.0	166.0	150.0	isopod	True	False
...	...	...	...	...	...	...	...	...	...	...	...	...	...
416	445	89	00	14	556.0	203.0	14.0	81.0	138.0	121.0	isopod	True	False
417	445	89	00	14	544.0	191.0	18.0	101.0	148.0	115.0	isopod	NaN	NaN
418	445	89	00	14	189.0	127.0	10.0	49.0	112.0	88.0	isopod	NaN	NaN
419	445	89	00	14	208.0	129.0	14.0	104.0	148.0	128.0	isopod	NaN	NaN
420	445	89	00	14	748.0	111.0	14.0	44.0	124.0	127.0	isopod	NaN	NaN

421 rows × 13 columns

Analyzing tracking results with trackpy

Now we can use the frame-wise coordinates to construct trajectories of isopods and fish. For this we will use the excellent trackpy library (http://soft-matter.github.io/trackpy/v0.4.2/). Trackpy is a Python package for particle tracking in 2D, 3D, and higher dimensions.

Here you need to find out what works best for your specific case - the larger search_range or memory are, the more challenging it is for the algorithm to find a solution, especially if you have many moving objects in your video. With only one, it should be ok to go to high values. The filtering step is optional, but can be useful to eliminate spurious trajectories.

## fish trajectories
fish_df = coordinates[coordinates['label'] == "fish" ] # just use the fish-coordinates
traj_fish = tp.link_df(fish_df,
                       search_range = 100, #how for to look for the fish in the next frame. can be large if fish swims fast
                       memory=60, # how long can the fish sit still
                       neighbor_strategy="KDTree",
                       link_strategy="nonrecursive")
traj_fish_filter = tp.filtering.filter_stubs(traj_fish,
                                             threshold=20) # filter out particles that were only found 20 times

## plot
plot = tp.plot_traj(traj_fish_filter,
                    superimpose=mt.image)
fig1 = plot.get_figure()
fig1.savefig(os.path.join(out_dir, mt.name + "_fish_trajectories.png"), dpi=300)

Frame 89: 1 trajectories present.

For isopods, we perform one additional step: because we know that we have 20 isopds, we remove frames with more particles.

## isopod trajectories
df_isopod = coordinates[coordinates['label'] == "isopod" ] # just use the isopod-coordinates
df_isopod_filtered = df_isopod.groupby("frame").filter(lambda x: len(x) < 20)  # filter out frames with too many points
traj_isopod = tp.link(df_isopod_filtered,
                      search_range = 50, # isopods are slow, so this can be small. especially important when using "multiple"
                      memory=200, # isopods can sit still longer
                      neighbor_strategy="KDTree",
                      link_strategy="nonrecursive")
traj_isopod_filter = tp.filtering.filter_stubs(traj_isopod,
                                               threshold=10) # each particle needs to be found at least 10 times

##p lot
plot = tp.plot_traj(traj_isopod_filter, superimpose=mt.image, colorby="particle")
fig1 = plot.get_figure()
fig1.savefig(os.path.join(out_dir, mt.name + "_isopod_trajectories.png"), dpi=300)

Frame 89: 5 trajectories present.

Finally we save all trajectories into one DataFrame and export them into to the specified directory, so you can do fine tuning in your favorite data analysis program (e.g. R or in Python with Pandas).

# save all
df = traj_isopod_filter.append(pd.DataFrame(data = traj_fish_filter), ignore_index=True)
df.to_csv(os.path.join(out_dir, mt.name + "_trajectories.csv"), sep=',')