Running PIE timelapse experiments

Setting up timelapse experiments

New setup files can also be created using the run_setup_wizard command, which walks the user through all the key parameters to enter one-by-one. This is recommended for users running new experiment types for the first time.

command-line

pie run_setup_wizard

Time-lapse PIE experiments require a single csv-format setup file to run. Some templates for experiment types commonly run in the Siegal lab can be found in sample_PIE_setup_files. Details on each setup file are provided below, and each can be used to analyze an experiment provided in the PIE_test_data/IN folder on github (see Sample Experiments).

General experimental setup

To set up a file for an experiment, modify an existing setup file. Setup files must be .csv files, and must have at least a ‘Parameter’ column for the name of the parameter being set, and a ‘Value’ column for its value.

The following are the minimal required parameters, and must be included in each experiment:

Required Setup File Parameters

Parameter	Explanation
output_path	Full path to folder from in which image analysis and growth rate outputs will be stored ACROSS ALL PHASES. Note: if your path contains backslash (’') characters (e.g. on Windows) you will need to use a double backslash instead (’\’) when specifying the path
input_path	Full path to folder from which images for this phase will be drawn (PHASE-SPECIFIC). Note: if your path contains backslash (’') characters (e.g. on Windows) you will need to use a double backslash instead (’\’) when specifying the path
im_format	Format in which images are saved; can be either ‘individual’ (i.e. where every timepoint-position-channel combination is saved to its own image) or ‘stack’ (i.e. where every image at a position is saved to a single file)
im_file_extension	File extension for image files in input_dir
main_channel_imagetype	Type of image in the main channel of this phase; can be ‘bright’ if cell bodies are brighter than background or ‘dark’ if they are darker than background (e.g. in phase contrast images)
label_order_list	Order in which timepoint position and channel are listed in filenames (separated by semicolons): e.g. channel;timepoint;position for image names like c2t01xy4030 or position;timepoint for image names like xy420t5
position_label_prefix	Label to be used for positions (e.g. ‘xy’ if position labels look like xy001); if left blank position label will not be included in filename
timepoint_label_prefix	Label to be used for timepoints (e.g. ‘t’ if position labels look like t01); if left blank timepoint label will not be included in filename
max_xy_position_num	Highest imaged xy position number
max_timepoint_num	Highest imaged timepoint number
timepoint_spacing	Spacing between timepoints (in seconds). Can be left blank to extract this information from file modification times; an integer for evenly spaced timepoints; or consecutive numbers separated by semicolons to provide the time at which each timepoint image was taken
main_channel_label	Label (e.g. ‘c1’) used by the microscope for the main channel being imaged from which cell segmentation occurs (i.e. the bright field or phase contrast channel); if blank channel label will not be included in filename
growth_window_timepoints	number of timepoints to include in growth rate calculation. PIE reports the highest growth rate for a colony calculated from growth_window_timepoints consecutive timepoints

The PIE github repository contains an example minimal setup file.

Filenames

Note that the values of many of these parameters determine the format of the filenames; see examples below for sample filenames and corresponding parameter values. Note that the number of digits in the time point and position is dependent on total_time point_num and max_xy_position_num, respectively.

Parameter	filename
	t1xy001c3.tif	pos1tp01.jpg
label_order_list	timepoint;position;channel	position;timepoint
position_label_prefix	xy	pos
timepoint_label_prefix	t	tp
main_channel_label	c3
max_xy_position_num	999	9
max_timepoint_num	9	99
im_file_extension	tif	jpg

Tip

The reliance on max_xy_position_num and max_timepoint_num to set the number of digits in the position and timepoint of the filename can be problematic for analysis of data that includes only a subset of the originally collected images. For example, if a user initially collected 100 timepoints worth of data, filenames are most likely t001-t100. However, if the user wants to wants to only analyze the first 99 of these timepoints and sets max_timepoint_num to 99, PIE will expect only two digits in the timepoint (t01-t99). This issue can be circumvented by adding any extra 0s to timepoint_label_prefix, i.e. setting it to ‘t0’ instead of ‘t’ in this example.

Running the experiment

To run a full image analysis experiment using PIE, you can use the run_timelapse_analysis function, which takes a setup file path as input:

command-linepython

pie run_timelapse_analysis /full/path/to/setup_file.csv

Note

Replace /full/path/to/setup_file.csv in the code above with the full path to the setup file for your experiment; also, see Sample Experiments. For file path format of Windows, MacOS or Unix, please refer to Examples in Running PIE single-image analysis.

Although many modifications to experiment setup and analysis can be made (see below), these changes are achieved by altering the setup file; all experiments can then be run using the code above.

Advanced analysis options

In addition to the default experiment processing parameters, a number of optional parameters can be altered that affect file processing, image analysis/colony recognition, and filtration of growth rates:

Additional processing options

The following additional options pertain to processing of your image files:

Additional general setup file parameters

Parameter	Default	Explanation
first_xy_position	1	First xy position at which time count starts (likely 1 or 0)
first_timepoint	1	First timepoint at which time count starts (likely 1 or 0)
extended_display_positions	1	list of imaging positions (as integers separated by semicolons) for which additional image processing info (eg threshold plots and colony mask boundaries overlaid on original images) are saved

Modifying image analysis

The following optional parameters allow users to modify how image analysis is performed:

Setup file parameters pertaining to image analysis

Parameter	Default	Explanation
hole_fill_area	Inf	Size (in pixels) of the largest empty space between cells (on the inside of a colony) to consider part of the colony. If set to Inf all intercell spaces within a colony are filled; if set to 0 no intercell spaces are filled. Intercell space is not typically easily identified in colonies of round cells; for these we recommend first trying the default setting (Inf). However intercell space can often be accurately parsed in colonies of rod-shaped cells
cleanup	FALSE	Whether to perform recursive cleanup step to remove pieces of background detected as part of the colony. This is helpful in certain imaging conditions but can result in parts of real colonies being missed; we recommend first trying analysis on a set of images without cleanup first. Images taken under identical imaging conditions should be consistent in whether or not they require cleanup.
max_proportion_exposed_edge	0.75	The max proportion of the edge of a detected colony ‘pie piece’ that is allowed to not touch a neighboring pie piece before being removed during cleanup step. Recommended to be set to 0.6-0.75. Only applies during cleanup steps.
cell_intensity_num	1	The number of distinct intensity categories belonging to cells in the image; determines whether 2 or 3 Gaussians are used to fit intensity log histogram for threshold determination. Can be set to 1 (default) or (in rare cases) 2.
perform_registration	TRUE	Whether to perform image registration between subsequent images in PhaseNum before determining colony tracking; recommended to be set to True except in cases when there is a small number of colonies (<5) or many spurious objects. Registration will only be performed between phases if perform_registration is set to True for every PhaseNum

Modifying growth rate filtration

The following optional parameters allow users to modify how growth rates measured for a time series experiment are filtered:

Setup file parameters pertaining to growth rate filtration

Parameter	Default	Explanation
minimum_growth_time	1	smallest number of timepoints in which an object needs to be independently tracked in order to be considered a colony for the purposes of determining distance between neighboring colonies
max_area_pixel_decrease	Inf	the maximum allowed decrease in area between subsequent timepoints (in pixels); all subsequent timepoints in the GR calculation after an area decrease greater than this amount are ignored (Inf for no cutoff)
max_area_fold_decrease	Inf	The cutoff fold-decrease in area between subsequent timepoints (as a fraction of total colony area); all subsequent timepoints in the growth rate calculation after an area decrease greater than this amount are ignored (Inf for no cutoff)
max_area_fold_increase	Inf	The cutoff fold-increase in area between subsequent timepoints (as a fraction of total colony area); all subsequent timepoints in the growth rate calculation after an area increase greater than this amount are ignored (Inf for no cutoff)
min_colony_area	0	The minimum area (in pixels) for a colony timepoint to be counted toward the GR calculation; must be at least 0
max_colony_area	Inf	The maximum area (in pixels) for a colony timepoint to be counted toward the GR calculation
min_correlation	-1	The minimum log linear correlation over all acceptable timepoints for the colony GR to be included in the population; setting to ~0.9 likely filters for eggregious image analysis issues at individual timepoints
min_foldX	-Inf	The minimum fold increase in area that a colony must have at some point within the growth window. Can be used to filter out e.g. dead cells
min_neighbor_dist	0	The minimum distance in pixels that a colony must be from its closest neighbor in order to be included in growth rate calculations; colonies at half this distance from the image edge will also be excluded. Because colonies that are close to each other tend to collide too early for their growth rates to be calculated having colonies be too close to each other at the start of the experiment will bias counting towards slow-growing colonies that don’t have a chance to collide before reaching enough timepoints to fulfill the growth_window_timepoints cutoff
max_colony_num	1000	The maximum number of colonies allowed in an image; images with more colonies detected will be treated as blank. This is an important failsafe in case thresholding fails. It should be an order of magnitude higher than the expected number of colonies. Setting this number to >10000 is not recommended as tracking images with that many colonies may result in out-of-memory errors

The PIE github repository contains an example setup file with non-default parameter values.

Adding fluorescent measurements

In experiments in which fluorescence data is collected alonside brightfield/phase contrast data, additional parameters must be provided in order for PIE to process the fluorescence data; note that all the default parameters here are empty, which results in a default of no fluorescence analysis being performed:

Setup file parameters pertaining to fluorescence measurements

Parameter	Default	Explanation
fluor_channel_scope_labels		Labels in the filename for each fluorescent channel being imaged (e.g. ‘c2’); separated by semicolons for each channel
fluor_channel_names		Names for each additional_channel_label as you want them to show up in the final output column labels (e.g. ‘RFP’ ‘mito_label’ etc); separated by semicolons for each channel
fluor_channel_thresholds		Thresholds to be used for removing saturated pixels in images (in absolute intensity); separated by semicolons for each channel
fluor_channel_timepoints		Timepoints for which fluorescent data is to be included in growth rate output; separated by semicolons for each channel. The options for this are: an integer corresponding to the timepoint number; ‘last_gr’ or ‘first_gr’ - the last or first timepoint for which a growth rate was measured; ‘last_tracked’ or ‘first_tracked’ - the last or first timepoint at which a colony was tracked; and ‘mean’ ‘median’ ‘min’ or ‘max’: summary statistic across all tracked timepoints

Colony outlines are always calculated based on a “main channel”, which should consist of either brightfield or phase contrast images; the colonies identified in the main channel will then be overlaid on any fluorescent images in the phase to calculate fluorescence levels.

We provide an example setup file with fluorescence data analysis in the PIE github repository.

Analysis of complex experiments

Phases

Each experiment may consist of one or more phases. A single phase consists of a single, continuous bout of imaging. PIE can analyze experiments consisting of multiple such phases. During growth rate analysis, growth rates will be calculated independently for any phase that contains multiple time points, but colony identities will be linked across phases. Multi-phase experiments are meant to allow users to continue to track the same colonies across multiple experimental treatments, with growth rate and lag reported independently for each.

To specify parameters for multiple experimental phases, add a PhaseNum column to your setup file. Phases must be consecutive integers (i.e. ‘1’, ‘2’, etc). For any parameters that differ between phases, the parameter must be specified for each phase on an individual line with its corresponding PhaseNum. For parameters that are common between experimental phases (e.g. output_path), PhaseNum may be set to ‘all’.

Because each phase of a multi-phase experiment should be imaged with the same set of imaging positions, and the outputs of all phases are collected in a single output folder, the values of the following parameters must be the same across all phases:

• output_path

• im_format

• first_xy_position

• max_xy_position_num

• extended_display_positions.

We provide an example two-phase setup file with fluorescence data analysis in the PIE github repository.

Post-phase fluorescent measurement and fluorescence-based classification

For experiments in which fluorescent and non-fluorescent strains (or strains with different fluorescent markers) are co-cultured, PIE can use fluorescence data to classify colonies by strain. It can be useful to collect this kind of ‘classification’ fluorescence data after an experiment (or experimental phase) is complete, to avoid spending time imaging in a fluorescent channel between each set of time points. Colony segmentation from brightfield or phase-contrast imaging in the previous phase can then be used to assign fluorescent values to colonies.

PIE allows for the creation of a special phase that includes only fluorescent images for a single time point, in which case the linked_phase parameter should be set to the phase number of the phase containing the brightfield/phase contrast data to be used for colony segmentation (we strongly recommend that this be the phase immediately before the fluorescent classification phase).

These ‘post-phase fluorescence’ phases require only a subset of parameters to be specified:

• linked_phase

• fluor_channel_scope_labels

• fluor_channel_names

• fluor_channel_thresholds

• fluor_channel_time points

• input_path

• first_xy_position

• extended_display_positions

• time point_label_prefix

• output_path

• im_file_extension

• label_order_list

• max_xy_position_num

• position_label_prefix

• im_format

We provide an example post-phase fluorescence setup file in the PIE github repository.