smsh5

Module for handling SMS data from HDF5 files

Bertus van Heerden and Joshua Botha University of Pretoria 2018

Module Contents

Classes

H5dataset	Represents an entire HDF5 dataset
Particle	Represents a particle in an H5dataset.
FakeCpts	Fake ChangePoints object for GlobalParticle.
GlobalParticle	Particle-like object containing levels from all particles.
Trace	Binned intensity trace.
Histogram	TCSPC histogram.
ParticleAllHists	Class containing all Histograms from Particle.
RasterScan	Class containing raster scan data.
Spectra	Class containing spectral data.

Attributes

__docformat__
logger

smsh5.__docformat__ = "'NumPy'"

smsh5.logger = "'smsh5'"

class smsh5.H5dataset(filename, sig_fb: processes.PassSigFeedback = None, prog_fb: processes.ProcessProgFeedback = None)

Represents an entire HDF5 dataset

Parameters:

• filename (str) – HDF5 file path

• sig_fb (PassSigFeedback, optional) – feedback queue for signals

• prog_fb (ProcessProgFeedback, optional) – feedback queue for updating progress bar

property file

HDF5 file.

cpa_has_run = 'False'

use_parallel = 'False'

name

_file

file_version

all_sums

all_raster_scans = '[]'

particles = '[]'

num_parts = '0'

channelwidth = 'None'

save_selected = 'None'

has_levels = 'False'

has_groups = 'False'

has_lifetimes = 'False'

irf = 'None'

irf_t = 'None'

has_irf = 'False'

has_spectra = 'False'

has_corr = 'False'

global_particle = 'None'

unload_file(should_close: bool = True, should_delete: bool = True)

Remove file reference and close and/or delete the file.

load_file(new_file: h5pickle.File)

Load pickled HDF file.

get_all_raster_scans(particle_names: List[str]) → list

Get all the raster scans from the file and handle appropriately.

Each raster scan has one or more particles associated with it. In the HDF5 file, this is manifested as having the same raster scan duplicated across particle objects. This function reads out all unique raster scans and assigns them each their particles. The return value is a map from particle numbers to ‘raster scan numbers - if there’s a raster scan for particles 1 and 2 and another one for particles 3-5, the map would be: [1, 1, 2, 2, 2]

Parameters:

particle_names (list[str]) – list of particle names to read raster scan data from

Returns:

map_particle_index – maps particle numbers to raster scan numbers.

Return type:

list or None

makehistograms(remove_zeros=True, startpoint=None, channel=True)

Put the (micro) arrival times into histograms.

bin_all_ints(binsize: float, sig_fb: processes.PassSigFeedback = None, prog_fb: processes.ProcessProgFeedback = None)

Bin the absolute times into intensity traces.

Parameters:

• binsize (float) – Time bin size in ms

• sig_fb (PassSigFeedback, optional) – feedback queue for signals

• prog_fb (ProcessProgFeedback, optional) – feedback queue for updating progress bar

save_particles(file_path, selected_nums: List[int])

Save selected particle to a new or existing HDF5 file.

Parameters:

• file_path (str) – Path to existing file, or to file that will be created.

• selected_nums (List[int]) – Particle numbers to be written to HDF5 file.

class smsh5.Particle(name: str, dataset_ind: int, dataset: H5dataset, raster_scan_dataset_index: int = None, raster_scan: RasterScan = None, is_secondary_part: bool = False, prim_part: Particle = None, sec_part: Particle = None, tmin=None, tmax=None, channelwidth=None, is_global: bool = False)

Represents a particle in an H5dataset.

Parameters:

• name (str) – The name of the particle

• dataset_ind (H5dataset) – The index of the particle in the dataset

• dataset (H5dataset) – The instance of the dataset to which this particle belongs

• raster_scan_dataset_index (int) – The index of the raster scan connected to the particle

• raster_scan (RasterScan) – The raster scan object this particle is connected to

• is_secondary_part (bool) – Whether this is a “secondary particle” that contains the data from a second TCSCPC card

• prim_part (Particle) – If this particle is a secondary particle, the corresponding primary particle

• sec_part (Particle) – If this particle is a primary particle, the corresponding secondary particle

• tmin (int, optional) – Minimum photon micro time in ns

• tmax (int, optional) – Maximum photon micro time in ns

• channelwidth (float, optional) – TCSPC histogram channelwidth in ns. Normally automatically determined.

• is_global (bool = False) – TODO: What is this?

property file

HDF5 file.

property file_group

The particle’s group in the HDF5 file.

property microtimes: h5pickle.Dataset

The particle’s microtimes.

property abstimes: h5pickle.Dataset

The particle’s absolute times.

property histogram: Histogram

The particle’s Histogram.

property use_roi_for_grouping: bool

Whether to use the particle’s ROI for grouping.

property grouped_with_roi: bool

Whether the particle’s ahca used ROI.

property roi_region_photon_inds: Tuple[int, int]

Indices of photons at ROI boundaries.

property num_photons_roi: int

Number of photons in the particle ROI.

property has_spectra: bool

Whether the particle has spectra.

property microtimes_roi: numpy.ndarray

Microtimes from the particle ROI.

property first_level_ind_in_roi

Index of the first level in the particle’s ROI.

property last_level_ind_in_roi

Index of the last level in the particle’s ROI.

property first_group_level_ind_in_roi

Index of the first grouped level in the particle’s ROI.

property last_group_level_ind_in_roi

Index of the last grouped level in the particle’s ROI.

property raster_scan_coordinates: tuple

The particle’s RS coordinates.

property has_levels

Whether the particle has levels.

property has_groups

Whether the particle has groups.

property has_corr

Whether the particle has a second-order correlation.

property ab_analysis

property groups: List[grouping.Group]

The particle’s grouped levels.

property num_groups

The number of level groups.

property groups_bounds

The particle’s groups’ bounds.

property groups_ints

The particle’s group intensities.

property grouping_bics

The particle’s group BIC’s.

property grouping_selected_ind

The currently selected group index.

property best_grouping_ind

The index of the grouping step with max BIC.

property grouping_num_groups

The number of groups in each AHCA step

property levels

The particle’s raw or grouped levels.

property levels_roi

The particle’s raw or grouped levels, from ROI.

property levels_roi_force

The particle’s raw levels, from ROI.

property group_levels: List[change_point.Level | grouping.GlobalLevel]

The particle’s grouped levels.

property group_levels_roi: List[change_point.Level]

The particle’s grouped levels, from ROI.

property num_levels

Number of raw or grouped particle levels.

property num_levels_roi

Number of raw particle levels in ROI.

property dwell_time_s

The particle’s total measurement time.

property dwell_time_roi

The particle’s ROI measurement time.

property level_ints

The particle’s raw or grouped level intensities.

property level_ints_roi

The particle’s ROI raw or grouped level intensities.

property level_dwelltimes

The particle’s raw or grouped level dwelltimes.

property level_dwelltimes_roi

The particle’s ROI raw or grouped level dwelltimes.

property has_burst: bool

Whether the particle has a ‘photon burst’.

property burst_levels: numpy.ndarray

Levels containing a ‘photon burst’.

property numexp

Number of exponents in fitted decay model.

property unique_name

Unique particle name in the case of dual-channel measurement.

property has_global_grouping: bool

Whether the H5dataset has global groups.

property global_particle: GlobalParticle

The H5dataset’s global particle.

property global_group_levels

The H5dataset’s globally grouped levels.

uuid

name

dataset

dataset_ind: int

file_version

set_histgram(histogram: Histogram) → None

set_histgram_roi(histogram: Histogram) → None

reset_grouping_ind()

Reset the grouping step selection.

remove_and_reset_grouping()

Re-initialize grouping and remove current data.

levels2data(use_grouped: bool = None, use_roi: bool = False, use_global_groups: bool = False) → Tuple[numpy.ndarray, numpy.ndarray]

Level plotting data.

Uses the Particle objects’ levels to generate two arrays for plotting the levels.

Parameters:

• use_grouped (bool, optional) – Whether to use grouped levels - if not provided, defaults to True if groups exist and false if they don’t.

• use_global_groups (bool = False) – whether to use globally grouped levels

• use_roi (bool = False) – whether to use ROI

Returns:

ints, times – Intensities as a function of time for plotting.

Return type:

Tuple[np.ndarray, np.ndarray]

lifetimes2data(use_grouped: bool = None, use_roi: bool = False) → Tuple[numpy.ndarray, numpy.ndarray]

Level lifetime plotting data.

Uses the Particle object’s levels to generate two arrays for plotting the level lifetimes.

Parameters:

• use_grouped (bool, optional) – Whether to use grouped levels - if not provided, defaults to True if groups exist and false if they don’t.

• use_roi (bool = False) – Whether to use ROI.

Returns:

lifetimes, times – Lifetime as a function of time for plotting.

Return type:

Tuple[np.ndarray, np.ndarray]

current2data(level_ind: int, use_roi: bool = False) → Tuple[numpy.ndarray, numpy.ndarray]

Current level plotting data.

Uses the Particle object’s selected level to generate two arrays for plotting level.

Parameters:

• level_ind (int) – Index of the level to be plotted.

• use_roi (bool = False) – Whether to use the ROI.

Returns:

levels_data, times – Intensity as a function of time for plotting.

Return type:

Tuple[np.ndarray, np.ndarray]

current_group2data(group_ind: int) → [numpy.ndarray, numpy.ndarray]

Current group plotting data.

Uses the Particle object’s selected group to generate two arrays for plotting the group.

Parameters:

group_ind (int) – Index of the level to be plotted.

Returns:

group_int, times – Intensity as a function of time for plotting.

Return type:

Tuple[np.ndarray, np.ndarray]

makehistograms(remove_zeros, startpoint, channel)

Make all histograms - whole trace and levels.

Parameters:

• remove_zeros (bool) – Whether to remove zeros at the start of the decay.

• startpoint (int) – Startpoint of the decay in number of time steps.

• channel (bool) – Whether to use the hardware channelwidth TODO: remove this parameter including downstream as it is never used

makehistogram(channel=True, add_roi: bool = False)

Put the arrival times into a histogram.

Parameters:

• channel (bool = True) – Whether to use the hardware channelwidth.

• add_roi (bool = True) – Whether to create the ROI Histogram as well.

makelevelhists(channel: bool = True, force_cpts_levels: bool = False, force_group_levels: bool = False)

Make level histograms.

Parameters:

• channel (bool = True) – Whether to use the hardware channelwidth.

• force_cpts_levels (bool = False) – Use self.cpts.levels instead of self.levels.

• force_group_levels (bool = False) – Use self.group_levels instead of self.levels.

makegrouplevelhists()

Make grouped level histograms.

makegrouphists(channel=True)

Make group histograms.

binints(binsize: int)

Bin the absolute times into intensity trace.

Parameters:

binsize (int) – Size of intensity trace time bins.

trim_trace(min_level_dwell_time: float, min_level_int: int, reset_roi: bool = True)

Trim the intensity trace.

This function trims the intensity trace to remove the photobleached end.

Parameters:

• min_level_dwell_time (float) – Minimum dwell time of bleached level for trimming.

• min_level_int (int) – Minimum intensity to classify level as not bleached.

• reset_roi (bool = True) – Whether to update the ROI to the trimmed area.

class smsh5.FakeCpts(num_levels: int, levels: list)

Fake ChangePoints object for GlobalParticle.

Parameters:

• num_levels (int) – number of intensity levels

• levels (List[GlobalLevel]) – list of global levels

num_levels

levels

num_cpts

has_levels = 'True'

class smsh5.GlobalParticle(particles: List[Particle], use_roi: bool = False)

Particle-like object containing levels from all particles.

Parameters:

• particles (List[Particle]) – Particles to include in global particle.

• use_roi (bool = False) – Whether to use ROI’s.

property has_levels

property has_groups

property groups

property num_groups

property groups_bounds

property groups_ints

property grouping_bics

property grouping_selected_ind

property best_grouping_ind

property grouping_num_groups

property level_particle_dataset_inds

property global_levels: list

is_global = 'True'

name = "'Global Particle'"

contributing_particles_dataset_inds

levels = '[]'

num_levels = '0'

num_levels_roi = '0'

dwell_time

num_photons

uuid

use_roi_for_grouping = 'False'

cpts = 'None'

ahca

run_grouping()

class smsh5.Trace(particle: Particle, binsize: int)

Binned intensity trace.

Parameters:

• particle (Particle) – The Particle which creates the Trace.

• binsize (int) – Size of time bin in ms.

binsize

intdata

inttimes

class smsh5.Histogram(particle: Particle, level: change_point.Level | List[int] = None, start_point: float = None, channel: bool = True, trim_start: bool = False, is_for_roi: bool = False)

TCSPC histogram.

This class represents histogrammed TCSPC arrival times (micro times) as well as multi-exponential fits thereof.

Parameters:

• particle (Particle) – The parent Particle of this object.

• level (Level or List = None) – The possible parent level of this object.

• start_point (float = None) – Start point for lifetime fit.

• channel (bool = True) – Whether to use hardware channel width.

• trim_start (bool = False) – Whether to trim zeros at the start of the histogram.

• is_for_roi (bool = False) – Whether this histogram is from a trace ROI.

property t

is_for_roi = 'False'

fitted_with_roi = 'None'

roi_region_used = 'None'

_particle

level = 'None'

original_kwargs

microtimes = 'None'

convd = 'None'

convd_t = 'None'

fitted = 'False'

fit_decay = 'None'

tau = 'None'

amp = 'None'

shift = 'None'

bg = 'None'

irfbg = 'None'

avtau = 'None'

numexp = 'None'

residuals = 'None'

fwhm = 'None'

stds = 'None'

avtaustd = 'None'

chisq = 'None'

dw = 'None'

dw_bound = 'None'

decay_roi_start_ns = 'None'

decay_roi_end_ns = 'None'

num_photons_used = 'None'

setup(level: change_point.Level | List[int] = None, start_point: float = None, channel: bool = True, trim_start: bool = False, use_roi: bool = False)

Set up the object.

This method can be called to re-do setup steps without reinitializing.

Parameters:

• level (Level or List = None) – The possible parent level of this object.

• start_point (float = None) – Start point for lifetime fit.

• channel (bool = True) – Whether to use hardware channel width.

• trim_start (bool = False) – Whether to trim zeros at the start of the histogram.

• use_roi (bool = False) – Whether this histogram is from a trace ROI.

update_roi()

Rerun setup to update ROI.

fit(numexp, tauparam, ampparam, shift, decaybg, irfbg, boundaries, addopt, irf, fwhm=None, normalize_amps=True, maximum_likelihood=False)

Fit a multiexponential decay to the histogram.

This function mainly calls the relevant code from tcspcfit.

Parameters:

• numexp (int) – Number of exponentials in fit function (1-3).

• tauparam (array_like) – Initial guess times (in ns). This is either in the format [tau1, tau2, …] or [[tau1, min1, max1, fix1], [tau2, …], …]. When the “fix” value is False, the min and max values are ignored.

• ampparam (array_like) – Initial guess amplitude. Format [amp1, amp2, …] or [[amp1, fix1], [amp2, fix2], …]

• shift (array_like) – Initial guess IRF shift. Either a float, or [shift, min, max, fix].

• decaybg (float) – Background value for decay. Will be estimated if not given.

• irfbg (float) – Background value for IRF. Will be estimated if not given.

• boundaries (list) – Start and end of fitting range as well as options for automatic determination of the parameters as used by FluoFit.calculate_boundaries.

• irf (ndarray) – Instrumental Response Function

• fwhm (float = None) – Full-width at half maximum of simulated irf. IRF is not simulated if fwhm is None.

• addopt (Dict = None) – Additional options for scipy.optimize.curve_fit (such as optimization parameters).

• normalize_amps (bool = True) – Whether to use normalized lifetime amplitudes.

• maximum_likelihood (bool = False) – Whether to use maximum likelihood fitting (otherwise use least squares).

levelhist(level)

static start_at_value(decay, t, neg_t=True, decaystart=None)

Helper method for setting decay startpoint.

class smsh5.ParticleAllHists(particle: Particle)

Class containing all Histograms from Particle.

Parameters:

particle (Particle) – The parent particle of this object.

part_uuid

numexp = 'None'

part_hist

has_level_hists

level_hists = '[]'

has_group_hists

group_hists = '[]'

fit_part_and_levels(channelwidth, start, end, fit_param: tcspcfit.FittingParameters)

Fit all Histograms in this object.

Parameters:

• channelwidth (float) – TCSPC channelwidth (time step size) in ns.

• start (int) – Fitting startpoint in number of time steps.

• end (int) – Fitting endpoint in number of time steps.

• fit_param (FittingParameters) – Object containing fit parameters.

class smsh5.RasterScan(h5dataset: H5dataset, particle_num: int, h5dataset_index: int, particle_indexes: List[int])

Class containing raster scan data.

A raster scan is a 2D intensity scan used to visualize particles before measurement.

Parameters:

• h5dataset (H5dataset) – The parent HDF5 dataset object.

• particle_num (int) – Number of particles connected to this raster scan.

• h5dataset_index (int) – Index of this raster scan in the dataset.

• particle_indexes (List[int]) – Dataset indices of the particles in this raster scan.

property dataset: h5pickle.Dataset

h5dataset

particle_num

h5dataset_index

particle_indexes

integration_time

pixel_per_line

range

x_start

y_start

x_axis_pos

y_axis_pos

class smsh5.Spectra(particle: Particle)

Class containing spectral data.

Spectra are recorded as a time scan, with a certain integration time over which a single spectrum is recorded using a grating and CCD.

Parameters:

particle (Particle) – The parent particle object.

property _has_spectra: bool

property data: h5pickle.Dataset

property wavelengths: numpy.ndarray

property series_times: numpy.ndarray

_particle