Example 4: Video analysis - predator prey interactions¶
This example features analysis of predator-prey interactions of one threespine stickleback (Gasterosteus aculeatus) and 20 benthic isopods (Asellus aquaticus). The goal is to extract movement trajectories of both species in response to each other, to see if isopods are reacting to the fish, and the other way round. At the same time, isopod pigmentation is measured in every frame to measure whether the fish has preferences for a particular phenotype (dark vs. light pigmentation, large bodied individuals).
The full video can be found here: https://vimeo.com/283075068
Fish movement - Fish movement is extracted by following the movement of the single biggest object.
Isopod movement - Movement of isopods over 15 seconds.
Video tracking with phenopype¶
[1]:
import os
import pandas as pd
import phenopype as pp
import trackpy as tp # can be installed using pip
[5]:
video_path = r"images/isopods_fish.mp4"
out_dir = r"../_temp/output/ex4"
First we initialize the motion tracker with our example video, and specify the video output.
[6]:
mt = pp.motion_tracker(video_path)
mt.video_output(save_suffix="v1", dirpath=out_dir)
--------------------------------------------------------------
Input video properties - "isopods_fish.mp4":
Frames per second: 29.97000099996667
N frames: 450
Length: 00:15 (mm:ss)
Dimensions: (960, 540)
Colour video: True
FourCC code: avc1
--------------------------------------------------------------
Save folder ../_temp/output/ex4 does not exist - create?.y
--------------------------------------------------------------
Output video settings - "isopods_fish.mp4":
Save name: isopods_fish_v1.mp4
Save dir: E:\git_repos\phenopype\_temp\output\ex4
Frames per second: 29.97000099996667
Dimensions: (960, 540)
Colour video: True
Format (FourCC code): avc1
--------------------------------------------------------------
Then we create a mask to only include the gravel area - this will exclude the walls where reflections and floating particles may result in false positives. A second mask in the center of the area can later be used to compare whether isopods tend to stay more on the side (shy behavior) or whether they move freely (bold behavior). In each captured frame there will be an entry for each isopods in which area it was detected.
Masking - on one hand we want to exclude unwanted areas, on the other hand, we want to see when isopds stays in the center or not.
[7]:
pp.preprocessing.create_mask(mt, label="full arena")
pp.preprocessing.create_mask(mt, label="center")
- create mask
Now we set up two tracking_methods
: the “fish” method is set to "single"
and will only capture the largest object, whereas the isopod method will capture all objects smaller than 30 pixels. You can play around with blur
and threshold
settings to see if you can get better results. The operations
argument is used to return specific information of the detected objects. We specify a few parameters to retrieve phenotypic information of the isopods. Later, we can then analyze which
phenotypes were foraged on which sediment background.
[8]:
fish = pp.tracking_method(label="fish", remove_shadows=True, min_length=30,
overlay_colour="red", mode="single",
blur=15, # bigger blurring kernel
threshold=200 #higher sensitivity
)
isopod = pp.tracking_method(label="isopod", remove_shadows=True, max_length=30,
overlay_colour="green", mode="multiple",
blur=9, # smaller blurring kernel
threshold=180, # lower sensitivity
operations=["diameter", # isopod size
"area", # isopod area
"grayscale", # isopod pigmentation
"grayscale_background"] # background darkness
)
Finally, we pass the methods on the detection settings, and also activate consecutive masking (c_mask=True
). This option will prohibit repeated detection of objects when several tracking methods are applied. For example, inside the detected fish-area sometimes isopod objects are detected, due to artifacts or incomplete subtraction results. Consecutive masking will “block” an area after the first method has been applied - the order by which methods are applied matters. The following
settings will create a rectangle shaped mask around the fish with a 200 pixel border.
[9]:
mt.detection_settings(methods=[fish, isopod],
c_mask=True,
c_mask_shape="rect",
c_mask_size=200)
{'blur_kernel': 15,
'label': 'fish',
'max_area': inf,
'max_length': inf,
'min_area': 0,
'min_length': 30,
'mode': 'single',
'operations': [],
'overlay_colour': (0, 0,
255),
'remove_shadows': True,
'threshold_value': 200}
{'blur_kernel': 9,
'label': 'isopod',
'max_area': inf,
'max_length': 30,
'min_area': 0,
'min_length': 0,
'mode': 'multiple',
'operations': ['diameter',
'area',
'grayscale',
'grayscale_background'],
'overlay_colour': (0, 255,
0),
'remove_shadows': True,
'threshold_value': 180}
--------------------------------------------------------------
Motion detection settings - "isopods_fish.mp4":
Background-subtractor: MOG
History: 60 seconds
Sensitivity: 10
Read every nth frame: 5
Detect shadows: True
Start after n seconds: 0
Finish after n seconds: -
--------------------------------------------------------------
[10]:
coordinates = mt.run_tracking()
Time: 00:00/00:15 - Frames: 1/450
Time: 00:00/00:15 - Frames: 2/450
Time: 00:00/00:15 - Frames: 3/450
Time: 00:00/00:15 - Frames: 4/450
Time: 00:00/00:15 - Frames: 5/450 - captured
Time: 00:00/00:15 - Frames: 6/450
Time: 00:00/00:15 - Frames: 7/450
Time: 00:00/00:15 - Frames: 8/450
Time: 00:00/00:15 - Frames: 9/450
Time: 00:00/00:15 - Frames: 10/450 - captured
Time: 00:00/00:15 - Frames: 11/450
Time: 00:00/00:15 - Frames: 12/450
Time: 00:00/00:15 - Frames: 13/450
Time: 00:00/00:15 - Frames: 14/450
Time: 00:00/00:15 - Frames: 15/450 - captured
Time: 00:00/00:15 - Frames: 16/450
Time: 00:00/00:15 - Frames: 17/450
Time: 00:00/00:15 - Frames: 18/450
Time: 00:00/00:15 - Frames: 19/450
Time: 00:00/00:15 - Frames: 20/450 - captured
Time: 00:00/00:15 - Frames: 21/450
Time: 00:00/00:15 - Frames: 22/450
Time: 00:00/00:15 - Frames: 23/450
Time: 00:00/00:15 - Frames: 24/450
Time: 00:00/00:15 - Frames: 25/450 - captured
Time: 00:00/00:15 - Frames: 26/450
Time: 00:00/00:15 - Frames: 27/450
Time: 00:00/00:15 - Frames: 28/450
Time: 00:00/00:15 - Frames: 29/450
Time: 00:01/00:15 - Frames: 30/450 - captured
Time: 00:01/00:15 - Frames: 31/450
Time: 00:01/00:15 - Frames: 32/450
Time: 00:01/00:15 - Frames: 33/450
Time: 00:01/00:15 - Frames: 34/450
Time: 00:01/00:15 - Frames: 35/450 - captured
Time: 00:01/00:15 - Frames: 36/450
Time: 00:01/00:15 - Frames: 37/450
Time: 00:01/00:15 - Frames: 38/450
Time: 00:01/00:15 - Frames: 39/450
Time: 00:01/00:15 - Frames: 40/450 - captured
Time: 00:01/00:15 - Frames: 41/450
Time: 00:01/00:15 - Frames: 42/450
Time: 00:01/00:15 - Frames: 43/450
Time: 00:01/00:15 - Frames: 44/450
Time: 00:01/00:15 - Frames: 45/450 - captured
Time: 00:01/00:15 - Frames: 46/450
Time: 00:01/00:15 - Frames: 47/450
Time: 00:01/00:15 - Frames: 48/450
Time: 00:01/00:15 - Frames: 49/450
Time: 00:01/00:15 - Frames: 50/450 - captured
Time: 00:01/00:15 - Frames: 51/450
Time: 00:01/00:15 - Frames: 52/450
Time: 00:01/00:15 - Frames: 53/450
Time: 00:01/00:15 - Frames: 54/450
Time: 00:01/00:15 - Frames: 55/450 - captured
Time: 00:01/00:15 - Frames: 56/450
Time: 00:01/00:15 - Frames: 57/450
Time: 00:01/00:15 - Frames: 58/450
Time: 00:01/00:15 - Frames: 59/450
Time: 00:02/00:15 - Frames: 60/450 - captured
Time: 00:02/00:15 - Frames: 61/450
Time: 00:02/00:15 - Frames: 62/450
Time: 00:02/00:15 - Frames: 63/450
Time: 00:02/00:15 - Frames: 64/450
Time: 00:02/00:15 - Frames: 65/450 - captured
Time: 00:02/00:15 - Frames: 66/450
Time: 00:02/00:15 - Frames: 67/450
Time: 00:02/00:15 - Frames: 68/450
Time: 00:02/00:15 - Frames: 69/450
Time: 00:02/00:15 - Frames: 70/450 - captured
Time: 00:02/00:15 - Frames: 71/450
Time: 00:02/00:15 - Frames: 72/450
Time: 00:02/00:15 - Frames: 73/450
Time: 00:02/00:15 - Frames: 74/450
Time: 00:02/00:15 - Frames: 75/450 - captured
Time: 00:02/00:15 - Frames: 76/450
Time: 00:02/00:15 - Frames: 77/450
Time: 00:02/00:15 - Frames: 78/450
Time: 00:02/00:15 - Frames: 79/450
Time: 00:02/00:15 - Frames: 80/450 - captured
Time: 00:02/00:15 - Frames: 81/450
Time: 00:02/00:15 - Frames: 82/450
Time: 00:02/00:15 - Frames: 83/450
Time: 00:02/00:15 - Frames: 84/450
Time: 00:02/00:15 - Frames: 85/450 - captured
Time: 00:02/00:15 - Frames: 86/450
Time: 00:02/00:15 - Frames: 87/450
Time: 00:02/00:15 - Frames: 88/450
Time: 00:02/00:15 - Frames: 89/450
Time: 00:03/00:15 - Frames: 90/450 - captured
Time: 00:03/00:15 - Frames: 91/450
Time: 00:03/00:15 - Frames: 92/450
Time: 00:03/00:15 - Frames: 93/450
Time: 00:03/00:15 - Frames: 94/450
Time: 00:03/00:15 - Frames: 95/450 - captured
Time: 00:03/00:15 - Frames: 96/450
Time: 00:03/00:15 - Frames: 97/450
Time: 00:03/00:15 - Frames: 98/450
Time: 00:03/00:15 - Frames: 99/450
Time: 00:03/00:15 - Frames: 100/450 - captured
Time: 00:03/00:15 - Frames: 101/450
Time: 00:03/00:15 - Frames: 102/450
Time: 00:03/00:15 - Frames: 103/450
Time: 00:03/00:15 - Frames: 104/450
Time: 00:03/00:15 - Frames: 105/450 - captured
Time: 00:03/00:15 - Frames: 106/450
Time: 00:03/00:15 - Frames: 107/450
Time: 00:03/00:15 - Frames: 108/450
Time: 00:03/00:15 - Frames: 109/450
Time: 00:03/00:15 - Frames: 110/450 - captured
Time: 00:03/00:15 - Frames: 111/450
Time: 00:03/00:15 - Frames: 112/450
Time: 00:03/00:15 - Frames: 113/450
Time: 00:03/00:15 - Frames: 114/450
Time: 00:03/00:15 - Frames: 115/450 - captured
Time: 00:03/00:15 - Frames: 116/450
Time: 00:03/00:15 - Frames: 117/450
Time: 00:03/00:15 - Frames: 118/450
Time: 00:03/00:15 - Frames: 119/450
Time: 00:04/00:15 - Frames: 120/450 - captured
Time: 00:04/00:15 - Frames: 121/450
Time: 00:04/00:15 - Frames: 122/450
Time: 00:04/00:15 - Frames: 123/450
Time: 00:04/00:15 - Frames: 124/450
Time: 00:04/00:15 - Frames: 125/450 - captured
Time: 00:04/00:15 - Frames: 126/450
Time: 00:04/00:15 - Frames: 127/450
Time: 00:04/00:15 - Frames: 128/450
Time: 00:04/00:15 - Frames: 129/450
Time: 00:04/00:15 - Frames: 130/450 - captured
Time: 00:04/00:15 - Frames: 131/450
Time: 00:04/00:15 - Frames: 132/450
Time: 00:04/00:15 - Frames: 133/450
Time: 00:04/00:15 - Frames: 134/450
Time: 00:04/00:15 - Frames: 135/450 - captured
Time: 00:04/00:15 - Frames: 136/450
Time: 00:04/00:15 - Frames: 137/450
Time: 00:04/00:15 - Frames: 138/450
Time: 00:04/00:15 - Frames: 139/450
Time: 00:04/00:15 - Frames: 140/450 - captured
Time: 00:04/00:15 - Frames: 141/450
Time: 00:04/00:15 - Frames: 142/450
Time: 00:04/00:15 - Frames: 143/450
Time: 00:04/00:15 - Frames: 144/450
Time: 00:04/00:15 - Frames: 145/450 - captured
Time: 00:04/00:15 - Frames: 146/450
Time: 00:04/00:15 - Frames: 147/450
Time: 00:04/00:15 - Frames: 148/450
Time: 00:04/00:15 - Frames: 149/450
Time: 00:05/00:15 - Frames: 150/450 - captured
Time: 00:05/00:15 - Frames: 151/450
Time: 00:05/00:15 - Frames: 152/450
Time: 00:05/00:15 - Frames: 153/450
Time: 00:05/00:15 - Frames: 154/450
Time: 00:05/00:15 - Frames: 155/450 - captured
Time: 00:05/00:15 - Frames: 156/450
Time: 00:05/00:15 - Frames: 157/450
Time: 00:05/00:15 - Frames: 158/450
Time: 00:05/00:15 - Frames: 159/450
Time: 00:05/00:15 - Frames: 160/450 - captured
Time: 00:05/00:15 - Frames: 161/450
Time: 00:05/00:15 - Frames: 162/450
Time: 00:05/00:15 - Frames: 163/450
Time: 00:05/00:15 - Frames: 164/450
Time: 00:05/00:15 - Frames: 165/450 - captured
Time: 00:05/00:15 - Frames: 166/450
Time: 00:05/00:15 - Frames: 167/450
Time: 00:05/00:15 - Frames: 168/450
Time: 00:05/00:15 - Frames: 169/450
Time: 00:05/00:15 - Frames: 170/450 - captured
Time: 00:05/00:15 - Frames: 171/450
Time: 00:05/00:15 - Frames: 172/450
Time: 00:05/00:15 - Frames: 173/450
Time: 00:05/00:15 - Frames: 174/450
Time: 00:05/00:15 - Frames: 175/450 - captured
Time: 00:05/00:15 - Frames: 176/450
Time: 00:05/00:15 - Frames: 177/450
Time: 00:05/00:15 - Frames: 178/450
Time: 00:05/00:15 - Frames: 179/450
Time: 00:06/00:15 - Frames: 180/450 - captured
Time: 00:06/00:15 - Frames: 181/450
Time: 00:06/00:15 - Frames: 182/450
Time: 00:06/00:15 - Frames: 183/450
Time: 00:06/00:15 - Frames: 184/450
Time: 00:06/00:15 - Frames: 185/450 - captured
Time: 00:06/00:15 - Frames: 186/450
Time: 00:06/00:15 - Frames: 187/450
Time: 00:06/00:15 - Frames: 188/450
Time: 00:06/00:15 - Frames: 189/450
Time: 00:06/00:15 - Frames: 190/450 - captured
Time: 00:06/00:15 - Frames: 191/450
Time: 00:06/00:15 - Frames: 192/450
Time: 00:06/00:15 - Frames: 193/450
Time: 00:06/00:15 - Frames: 194/450
Time: 00:06/00:15 - Frames: 195/450 - captured
Time: 00:06/00:15 - Frames: 196/450
Time: 00:06/00:15 - Frames: 197/450
Time: 00:06/00:15 - Frames: 198/450
Time: 00:06/00:15 - Frames: 199/450
Time: 00:06/00:15 - Frames: 200/450 - captured
Time: 00:06/00:15 - Frames: 201/450
Time: 00:06/00:15 - Frames: 202/450
Time: 00:06/00:15 - Frames: 203/450
Time: 00:06/00:15 - Frames: 204/450
Time: 00:06/00:15 - Frames: 205/450 - captured
Time: 00:06/00:15 - Frames: 206/450
Time: 00:06/00:15 - Frames: 207/450
Time: 00:06/00:15 - Frames: 208/450
Time: 00:06/00:15 - Frames: 209/450
Time: 00:07/00:15 - Frames: 210/450 - captured
Time: 00:07/00:15 - Frames: 211/450
Time: 00:07/00:15 - Frames: 212/450
Time: 00:07/00:15 - Frames: 213/450
Time: 00:07/00:15 - Frames: 214/450
Time: 00:07/00:15 - Frames: 215/450 - captured
Time: 00:07/00:15 - Frames: 216/450
Time: 00:07/00:15 - Frames: 217/450
Time: 00:07/00:15 - Frames: 218/450
Time: 00:07/00:15 - Frames: 219/450
Time: 00:07/00:15 - Frames: 220/450 - captured
Time: 00:07/00:15 - Frames: 221/450
Time: 00:07/00:15 - Frames: 222/450
Time: 00:07/00:15 - Frames: 223/450
Time: 00:07/00:15 - Frames: 224/450
Time: 00:07/00:15 - Frames: 225/450 - captured
Time: 00:07/00:15 - Frames: 226/450
Time: 00:07/00:15 - Frames: 227/450
Time: 00:07/00:15 - Frames: 228/450
Time: 00:07/00:15 - Frames: 229/450
Time: 00:07/00:15 - Frames: 230/450 - captured
Time: 00:07/00:15 - Frames: 231/450
Time: 00:07/00:15 - Frames: 232/450
Time: 00:07/00:15 - Frames: 233/450
Time: 00:07/00:15 - Frames: 234/450
Time: 00:07/00:15 - Frames: 235/450 - captured
Time: 00:07/00:15 - Frames: 236/450
Time: 00:07/00:15 - Frames: 237/450
Time: 00:07/00:15 - Frames: 238/450
Time: 00:07/00:15 - Frames: 239/450
Time: 00:08/00:15 - Frames: 240/450 - captured
Time: 00:08/00:15 - Frames: 241/450
Time: 00:08/00:15 - Frames: 242/450
Time: 00:08/00:15 - Frames: 243/450
Time: 00:08/00:15 - Frames: 244/450
Time: 00:08/00:15 - Frames: 245/450 - captured
Time: 00:08/00:15 - Frames: 246/450
Time: 00:08/00:15 - Frames: 247/450
Time: 00:08/00:15 - Frames: 248/450
Time: 00:08/00:15 - Frames: 249/450
Time: 00:08/00:15 - Frames: 250/450 - captured
Time: 00:08/00:15 - Frames: 251/450
Time: 00:08/00:15 - Frames: 252/450
Time: 00:08/00:15 - Frames: 253/450
Time: 00:08/00:15 - Frames: 254/450
Time: 00:08/00:15 - Frames: 255/450 - captured
Time: 00:08/00:15 - Frames: 256/450
Time: 00:08/00:15 - Frames: 257/450
Time: 00:08/00:15 - Frames: 258/450
Time: 00:08/00:15 - Frames: 259/450
Time: 00:08/00:15 - Frames: 260/450 - captured
Time: 00:08/00:15 - Frames: 261/450
Time: 00:08/00:15 - Frames: 262/450
Time: 00:08/00:15 - Frames: 263/450
Time: 00:08/00:15 - Frames: 264/450
Time: 00:08/00:15 - Frames: 265/450 - captured
Time: 00:08/00:15 - Frames: 266/450
Time: 00:08/00:15 - Frames: 267/450
Time: 00:08/00:15 - Frames: 268/450
Time: 00:08/00:15 - Frames: 269/450
Time: 00:09/00:15 - Frames: 270/450 - captured
Time: 00:09/00:15 - Frames: 271/450
Time: 00:09/00:15 - Frames: 272/450
Time: 00:09/00:15 - Frames: 273/450
Time: 00:09/00:15 - Frames: 274/450
Time: 00:09/00:15 - Frames: 275/450 - captured
Time: 00:09/00:15 - Frames: 276/450
Time: 00:09/00:15 - Frames: 277/450
Time: 00:09/00:15 - Frames: 278/450
Time: 00:09/00:15 - Frames: 279/450
Time: 00:09/00:15 - Frames: 280/450 - captured
Time: 00:09/00:15 - Frames: 281/450
Time: 00:09/00:15 - Frames: 282/450
Time: 00:09/00:15 - Frames: 283/450
Time: 00:09/00:15 - Frames: 284/450
Time: 00:09/00:15 - Frames: 285/450 - captured
Time: 00:09/00:15 - Frames: 286/450
Time: 00:09/00:15 - Frames: 287/450
Time: 00:09/00:15 - Frames: 288/450
Time: 00:09/00:15 - Frames: 289/450
Time: 00:09/00:15 - Frames: 290/450 - captured
Time: 00:09/00:15 - Frames: 291/450
Time: 00:09/00:15 - Frames: 292/450
Time: 00:09/00:15 - Frames: 293/450
Time: 00:09/00:15 - Frames: 294/450
Time: 00:09/00:15 - Frames: 295/450 - captured
Time: 00:09/00:15 - Frames: 296/450
Time: 00:09/00:15 - Frames: 297/450
Time: 00:09/00:15 - Frames: 298/450
Time: 00:09/00:15 - Frames: 299/450
Time: 00:10/00:15 - Frames: 300/450 - captured
Time: 00:10/00:15 - Frames: 301/450
Time: 00:10/00:15 - Frames: 302/450
Time: 00:10/00:15 - Frames: 303/450
Time: 00:10/00:15 - Frames: 304/450
Time: 00:10/00:15 - Frames: 305/450 - captured
Time: 00:10/00:15 - Frames: 306/450
Time: 00:10/00:15 - Frames: 307/450
Time: 00:10/00:15 - Frames: 308/450
Time: 00:10/00:15 - Frames: 309/450
Time: 00:10/00:15 - Frames: 310/450 - captured
Time: 00:10/00:15 - Frames: 311/450
Time: 00:10/00:15 - Frames: 312/450
Time: 00:10/00:15 - Frames: 313/450
Time: 00:10/00:15 - Frames: 314/450
Time: 00:10/00:15 - Frames: 315/450 - captured
Time: 00:10/00:15 - Frames: 316/450
Time: 00:10/00:15 - Frames: 317/450
Time: 00:10/00:15 - Frames: 318/450
Time: 00:10/00:15 - Frames: 319/450
Time: 00:10/00:15 - Frames: 320/450 - captured
Time: 00:10/00:15 - Frames: 321/450
Time: 00:10/00:15 - Frames: 322/450
Time: 00:10/00:15 - Frames: 323/450
Time: 00:10/00:15 - Frames: 324/450
Time: 00:10/00:15 - Frames: 325/450 - captured
Time: 00:10/00:15 - Frames: 326/450
Time: 00:10/00:15 - Frames: 327/450
Time: 00:10/00:15 - Frames: 328/450
Time: 00:10/00:15 - Frames: 329/450
Time: 00:11/00:15 - Frames: 330/450 - captured
Time: 00:11/00:15 - Frames: 331/450
Time: 00:11/00:15 - Frames: 332/450
Time: 00:11/00:15 - Frames: 333/450
Time: 00:11/00:15 - Frames: 334/450
Time: 00:11/00:15 - Frames: 335/450 - captured
Time: 00:11/00:15 - Frames: 336/450
Time: 00:11/00:15 - Frames: 337/450
Time: 00:11/00:15 - Frames: 338/450
Time: 00:11/00:15 - Frames: 339/450
Time: 00:11/00:15 - Frames: 340/450 - captured
Time: 00:11/00:15 - Frames: 341/450
Time: 00:11/00:15 - Frames: 342/450
Time: 00:11/00:15 - Frames: 343/450
Time: 00:11/00:15 - Frames: 344/450
Time: 00:11/00:15 - Frames: 345/450 - captured
Time: 00:11/00:15 - Frames: 346/450
Time: 00:11/00:15 - Frames: 347/450
Time: 00:11/00:15 - Frames: 348/450
Time: 00:11/00:15 - Frames: 349/450
Time: 00:11/00:15 - Frames: 350/450 - captured
Time: 00:11/00:15 - Frames: 351/450
Time: 00:11/00:15 - Frames: 352/450
Time: 00:11/00:15 - Frames: 353/450
Time: 00:11/00:15 - Frames: 354/450
Time: 00:11/00:15 - Frames: 355/450 - captured
Time: 00:11/00:15 - Frames: 356/450
Time: 00:11/00:15 - Frames: 357/450
Time: 00:11/00:15 - Frames: 358/450
Time: 00:11/00:15 - Frames: 359/450
Time: 00:12/00:15 - Frames: 360/450 - captured
Time: 00:12/00:15 - Frames: 361/450
Time: 00:12/00:15 - Frames: 362/450
Time: 00:12/00:15 - Frames: 363/450
Time: 00:12/00:15 - Frames: 364/450
Time: 00:12/00:15 - Frames: 365/450 - captured
Time: 00:12/00:15 - Frames: 366/450
Time: 00:12/00:15 - Frames: 367/450
Time: 00:12/00:15 - Frames: 368/450
Time: 00:12/00:15 - Frames: 369/450
Time: 00:12/00:15 - Frames: 370/450 - captured
Time: 00:12/00:15 - Frames: 371/450
Time: 00:12/00:15 - Frames: 372/450
Time: 00:12/00:15 - Frames: 373/450
Time: 00:12/00:15 - Frames: 374/450
Time: 00:12/00:15 - Frames: 375/450 - captured
Time: 00:12/00:15 - Frames: 376/450
Time: 00:12/00:15 - Frames: 377/450
Time: 00:12/00:15 - Frames: 378/450
Time: 00:12/00:15 - Frames: 379/450
Time: 00:12/00:15 - Frames: 380/450 - captured
Time: 00:12/00:15 - Frames: 381/450
Time: 00:12/00:15 - Frames: 382/450
Time: 00:12/00:15 - Frames: 383/450
Time: 00:12/00:15 - Frames: 384/450
Time: 00:12/00:15 - Frames: 385/450 - captured
Time: 00:12/00:15 - Frames: 386/450
Time: 00:12/00:15 - Frames: 387/450
Time: 00:12/00:15 - Frames: 388/450
Time: 00:12/00:15 - Frames: 389/450
Time: 00:13/00:15 - Frames: 390/450 - captured
Time: 00:13/00:15 - Frames: 391/450
Time: 00:13/00:15 - Frames: 392/450
Time: 00:13/00:15 - Frames: 393/450
Time: 00:13/00:15 - Frames: 394/450
Time: 00:13/00:15 - Frames: 395/450 - captured
Time: 00:13/00:15 - Frames: 396/450
Time: 00:13/00:15 - Frames: 397/450
Time: 00:13/00:15 - Frames: 398/450
Time: 00:13/00:15 - Frames: 399/450
Time: 00:13/00:15 - Frames: 400/450 - captured
Time: 00:13/00:15 - Frames: 401/450
Time: 00:13/00:15 - Frames: 402/450
Time: 00:13/00:15 - Frames: 403/450
Time: 00:13/00:15 - Frames: 404/450
Time: 00:13/00:15 - Frames: 405/450 - captured
Time: 00:13/00:15 - Frames: 406/450
Time: 00:13/00:15 - Frames: 407/450
Time: 00:13/00:15 - Frames: 408/450
Time: 00:13/00:15 - Frames: 409/450
Time: 00:13/00:15 - Frames: 410/450 - captured
Time: 00:13/00:15 - Frames: 411/450
Time: 00:13/00:15 - Frames: 412/450
Time: 00:13/00:15 - Frames: 413/450
Time: 00:13/00:15 - Frames: 414/450
Time: 00:13/00:15 - Frames: 415/450 - captured
Time: 00:13/00:15 - Frames: 416/450
Time: 00:13/00:15 - Frames: 417/450
Time: 00:13/00:15 - Frames: 418/450
Time: 00:13/00:15 - Frames: 419/450
Time: 00:14/00:15 - Frames: 420/450 - captured
Time: 00:14/00:15 - Frames: 421/450
Time: 00:14/00:15 - Frames: 422/450
Time: 00:14/00:15 - Frames: 423/450
Time: 00:14/00:15 - Frames: 424/450
Time: 00:14/00:15 - Frames: 425/450 - captured
Time: 00:14/00:15 - Frames: 426/450
Time: 00:14/00:15 - Frames: 427/450
Time: 00:14/00:15 - Frames: 428/450
Time: 00:14/00:15 - Frames: 429/450
Time: 00:14/00:15 - Frames: 430/450 - captured
Time: 00:14/00:15 - Frames: 431/450
Time: 00:14/00:15 - Frames: 432/450
Time: 00:14/00:15 - Frames: 433/450
Time: 00:14/00:15 - Frames: 434/450
Time: 00:14/00:15 - Frames: 435/450 - captured
Time: 00:14/00:15 - Frames: 436/450
Time: 00:14/00:15 - Frames: 437/450
Time: 00:14/00:15 - Frames: 438/450
Time: 00:14/00:15 - Frames: 439/450
Time: 00:14/00:15 - Frames: 440/450 - captured
Time: 00:14/00:15 - Frames: 441/450
Time: 00:14/00:15 - Frames: 442/450
Time: 00:14/00:15 - Frames: 443/450
Time: 00:14/00:15 - Frames: 444/450
Time: 00:14/00:15 - Frames: 445/450 - captured
Time: 00:14/00:15 - Frames: 446/450
Time: 00:14/00:15 - Frames: 447/450
Time: 00:14/00:15 - Frames: 448/450
After completing the tracking, we end up with a big data frame of all contours
[11]:
coordinates.to_csv(os.path.join(out_dir, mt.name + "coordinates.csv"), sep=',')
coordinates
[11]:
frame_abs | frame | mins | secs | x | y | diameter | area | grayscale | grayscale_background | label | full arena | center | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | 30 | 6 | 00 | 01 | 608.0 | 345.0 | 7.0 | 23.0 | 136.0 | 130.0 | isopod | True | True |
1 | 30 | 6 | 00 | 01 | 546.0 | 249.0 | 11.0 | 54.0 | 116.0 | 106.0 | isopod | NaN | NaN |
2 | 35 | 7 | 00 | 01 | 575.0 | 306.0 | 8.0 | 47.0 | 125.0 | 117.0 | isopod | True | True |
3 | 45 | 9 | 00 | 01 | 193.0 | 305.0 | 5.0 | 13.0 | 185.0 | 173.0 | isopod | True | False |
4 | 50 | 10 | 00 | 01 | 193.0 | 308.0 | 7.0 | 27.0 | 166.0 | 150.0 | isopod | True | False |
... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
416 | 445 | 89 | 00 | 14 | 556.0 | 203.0 | 14.0 | 81.0 | 138.0 | 121.0 | isopod | True | False |
417 | 445 | 89 | 00 | 14 | 544.0 | 191.0 | 18.0 | 101.0 | 148.0 | 115.0 | isopod | NaN | NaN |
418 | 445 | 89 | 00 | 14 | 189.0 | 127.0 | 10.0 | 49.0 | 112.0 | 88.0 | isopod | NaN | NaN |
419 | 445 | 89 | 00 | 14 | 208.0 | 129.0 | 14.0 | 104.0 | 148.0 | 128.0 | isopod | NaN | NaN |
420 | 445 | 89 | 00 | 14 | 748.0 | 111.0 | 14.0 | 44.0 | 124.0 | 127.0 | isopod | NaN | NaN |
421 rows × 13 columns
Analyzing tracking results with trackpy
¶
Now we can use the frame-wise coordinates to construct trajectories of isopods and fish. For this we will use the excellent trackpy
library (http://soft-matter.github.io/trackpy/v0.4.2/). Trackpy is a Python package for particle tracking in 2D, 3D, and higher dimensions.
Here you need to find out what works best for your specific case - the larger search_range or memory are, the more challenging it is for the algorithm to find a solution, especially if you have many moving objects in your video. With only one, it should be ok to go to high values. The filtering step is optional, but can be useful to eliminate spurious trajectories.
[12]:
## fish trajectories
fish_df = coordinates[coordinates['label'] == "fish" ] # just use the fish-coordinates
traj_fish = tp.link_df(fish_df,
search_range = 100, #how for to look for the fish in the next frame. can be large if fish swims fast
memory=60, # how long can the fish sit still
neighbor_strategy="KDTree",
link_strategy="nonrecursive")
traj_fish_filter = tp.filtering.filter_stubs(traj_fish,
threshold=20) # filter out particles that were only found 20 times
## plot
plot = tp.plot_traj(traj_fish_filter,
superimpose=mt.image)
fig1 = plot.get_figure()
fig1.savefig(os.path.join(out_dir, mt.name + "_fish_trajectories.png"), dpi=300)
Frame 89: 1 trajectories present.

For isopods, we perform one additional step: because we know that we have 20 isopds, we remove frames with more particles.
[13]:
## isopod trajectories
df_isopod = coordinates[coordinates['label'] == "isopod" ] # just use the isopod-coordinates
df_isopod_filtered = df_isopod.groupby("frame").filter(lambda x: len(x) < 20) # filter out frames with too many points
traj_isopod = tp.link(df_isopod_filtered,
search_range = 50, # isopods are slow, so this can be small. especially important when using "multiple"
memory=200, # isopods can sit still longer
neighbor_strategy="KDTree",
link_strategy="nonrecursive")
traj_isopod_filter = tp.filtering.filter_stubs(traj_isopod,
threshold=10) # each particle needs to be found at least 10 times
##p lot
plot = tp.plot_traj(traj_isopod_filter, superimpose=mt.image, colorby="particle")
fig1 = plot.get_figure()
fig1.savefig(os.path.join(out_dir, mt.name + "_isopod_trajectories.png"), dpi=300)
Frame 89: 5 trajectories present.

Finally we save all trajectories into one DataFrame and export them into to the specified directory, so you can do fine tuning in your favorite data analysis program (e.g. R or in Python with Pandas).
[15]:
# save all
df = traj_isopod_filter.append(pd.DataFrame(data = traj_fish_filter), ignore_index=True)
df.to_csv(os.path.join(out_dir, mt.name + "_trajectories.csv"), sep=',')
[ ]: