:py:mod:`smsh5` =============== .. py:module:: smsh5 .. autoapi-nested-parse:: Module for handling SMS data from HDF5 files Bertus van Heerden and Joshua Botha University of Pretoria 2018 Module Contents --------------- Classes ~~~~~~~ .. autoapisummary:: smsh5.H5dataset smsh5.Particle smsh5.FakeCpts smsh5.GlobalParticle smsh5.Trace smsh5.Histogram smsh5.ParticleAllHists smsh5.RasterScan smsh5.Spectra Attributes ~~~~~~~~~~ .. autoapisummary:: smsh5.__docformat__ smsh5.logger .. py:data:: __docformat__ :value: "'NumPy'" .. py:data:: logger :value: "'smsh5'" .. py:class:: H5dataset(filename, sig_fb: processes.PassSigFeedback = None, prog_fb: processes.ProcessProgFeedback = None) Represents an entire HDF5 dataset :param filename: HDF5 file path :type filename: str :param sig_fb: feedback queue for signals :type sig_fb: PassSigFeedback, optional :param prog_fb: feedback queue for updating progress bar :type prog_fb: ProcessProgFeedback, optional .. py:property:: file HDF5 file. .. py:attribute:: cpa_has_run :value: 'False' .. py:attribute:: use_parallel :value: 'False' .. py:attribute:: name .. py:attribute:: _file .. py:attribute:: file_version .. py:attribute:: all_sums .. py:attribute:: all_raster_scans :value: '[]' .. py:attribute:: particles :value: '[]' .. py:attribute:: num_parts :value: '0' .. py:attribute:: channelwidth :value: 'None' .. py:attribute:: save_selected :value: 'None' .. py:attribute:: has_levels :value: 'False' .. py:attribute:: has_groups :value: 'False' .. py:attribute:: has_lifetimes :value: 'False' .. py:attribute:: irf :value: 'None' .. py:attribute:: irf_t :value: 'None' .. py:attribute:: has_irf :value: 'False' .. py:attribute:: has_spectra :value: 'False' .. py:attribute:: has_corr :value: 'False' .. py:attribute:: global_particle :value: 'None' .. py:method:: unload_file(should_close: bool = True, should_delete: bool = True) Remove file reference and close and/or delete the file. .. py:method:: load_file(new_file: h5pickle.File) Load pickled HDF file. .. py:method:: get_all_raster_scans(particle_names: List[str]) -> list Get all the raster scans from the file and handle appropriately. Each raster scan has one or more particles associated with it. In the HDF5 file, this is manifested as having the same raster scan duplicated across particle objects. This function reads out all unique raster scans and assigns them each their particles. The return value is a map from particle numbers to 'raster scan numbers - if there's a raster scan for particles 1 and 2 and another one for particles 3-5, the map would be: [1, 1, 2, 2, 2] :param particle_names: list of particle names to read raster scan data from :type particle_names: list[str] :returns: **map_particle_index** -- maps particle numbers to raster scan numbers. :rtype: list or None .. py:method:: makehistograms(remove_zeros=True, startpoint=None, channel=True) Put the (micro) arrival times into histograms. .. py:method:: bin_all_ints(binsize: float, sig_fb: processes.PassSigFeedback = None, prog_fb: processes.ProcessProgFeedback = None) Bin the absolute times into intensity traces. :param binsize: Time bin size in ms :type binsize: float :param sig_fb: feedback queue for signals :type sig_fb: PassSigFeedback, optional :param prog_fb: feedback queue for updating progress bar :type prog_fb: ProcessProgFeedback, optional .. py:method:: save_particles(file_path, selected_nums: List[int]) Save selected particle to a new or existing HDF5 file. :param file_path: Path to existing file, or to file that will be created. :type file_path: str :param selected_nums: Particle numbers to be written to HDF5 file. :type selected_nums: List[int] .. py:class:: Particle(name: str, dataset_ind: int, dataset: H5dataset, raster_scan_dataset_index: int = None, raster_scan: RasterScan = None, is_secondary_part: bool = False, prim_part: Particle = None, sec_part: Particle = None, tmin=None, tmax=None, channelwidth=None, is_global: bool = False) Represents a particle in an `H5dataset`. :param name: The name of the particle :type name: str :param dataset_ind: The index of the particle in the dataset :type dataset_ind: H5dataset :param dataset: The instance of the dataset to which this particle belongs :type dataset: H5dataset :param raster_scan_dataset_index: The index of the raster scan connected to the particle :type raster_scan_dataset_index: int :param raster_scan: The raster scan object this particle is connected to :type raster_scan: RasterScan :param is_secondary_part: Whether this is a "secondary particle" that contains the data from a second TCSCPC card :type is_secondary_part: bool :param prim_part: If this particle is a secondary particle, the corresponding primary particle :type prim_part: Particle :param sec_part: If this particle is a primary particle, the corresponding secondary particle :type sec_part: Particle :param tmin: Minimum photon micro time in ns :type tmin: int, optional :param tmax: Maximum photon micro time in ns :type tmax: int, optional :param channelwidth: TCSPC histogram channelwidth in ns. Normally automatically determined. :type channelwidth: float, optional :param is_global: TODO: What is this? :type is_global: bool = False .. py:property:: file HDF5 file. .. py:property:: file_group The particle's group in the HDF5 file. .. py:property:: microtimes :type: h5pickle.Dataset The particle's microtimes. .. py:property:: abstimes :type: h5pickle.Dataset The particle's absolute times. .. py:property:: histogram :type: Histogram The particle's `Histogram`. .. py:property:: use_roi_for_grouping :type: bool Whether to use the particle's ROI for grouping. .. py:property:: grouped_with_roi :type: bool Whether the particle's `ahca` used ROI. .. py:property:: roi_region_photon_inds :type: Tuple[int, int] Indices of photons at ROI boundaries. .. py:property:: num_photons_roi :type: int Number of photons in the particle ROI. .. py:property:: has_spectra :type: bool Whether the particle has spectra. .. py:property:: microtimes_roi :type: numpy.ndarray Microtimes from the particle ROI. .. py:property:: first_level_ind_in_roi Index of the first level in the particle's ROI. .. py:property:: last_level_ind_in_roi Index of the last level in the particle's ROI. .. py:property:: first_group_level_ind_in_roi Index of the first grouped level in the particle's ROI. .. py:property:: last_group_level_ind_in_roi Index of the last grouped level in the particle's ROI. .. py:property:: raster_scan_coordinates :type: tuple The particle's RS coordinates. .. py:property:: has_levels Whether the particle has levels. .. py:property:: has_groups Whether the particle has groups. .. py:property:: has_corr Whether the particle has a second-order correlation. .. py:property:: ab_analysis .. py:property:: groups :type: List[grouping.Group] The particle's grouped levels. .. py:property:: num_groups The number of level groups. .. py:property:: groups_bounds The particle's groups' bounds. .. py:property:: groups_ints The particle's group intensities. .. py:property:: grouping_bics The particle's group BIC's. .. py:property:: grouping_selected_ind The currently selected group index. .. py:property:: best_grouping_ind The index of the grouping step with max BIC. .. py:property:: grouping_num_groups The number of groups in each AHCA step .. py:property:: levels The particle's raw or grouped levels. .. py:property:: levels_roi The particle's raw or grouped levels, from ROI. .. py:property:: levels_roi_force The particle's raw levels, from ROI. .. py:property:: group_levels :type: List[Union[change_point.Level, grouping.GlobalLevel]] The particle's grouped levels. .. py:property:: group_levels_roi :type: List[change_point.Level] The particle's grouped levels, from ROI. .. py:property:: num_levels Number of raw or grouped particle levels. .. py:property:: num_levels_roi Number of raw particle levels in ROI. .. py:property:: dwell_time_s The particle's total measurement time. .. py:property:: dwell_time_roi The particle's ROI measurement time. .. py:property:: level_ints The particle's raw or grouped level intensities. .. py:property:: level_ints_roi The particle's ROI raw or grouped level intensities. .. py:property:: level_dwelltimes The particle's raw or grouped level dwelltimes. .. py:property:: level_dwelltimes_roi The particle's ROI raw or grouped level dwelltimes. .. py:property:: has_burst :type: bool Whether the particle has a 'photon burst'. .. py:property:: burst_levels :type: numpy.ndarray Levels containing a 'photon burst'. .. py:property:: numexp Number of exponents in fitted decay model. .. py:property:: unique_name Unique particle name in the case of dual-channel measurement. .. py:property:: has_global_grouping :type: bool Whether the H5dataset has global groups. .. py:property:: global_particle :type: GlobalParticle The H5dataset's global particle. .. py:property:: global_group_levels The H5dataset's globally grouped levels. .. py:attribute:: uuid .. py:attribute:: name .. py:attribute:: dataset .. py:attribute:: dataset_ind :type: int .. py:attribute:: file_version .. py:method:: set_histgram(histogram: Histogram) -> None .. py:method:: set_histgram_roi(histogram: Histogram) -> None .. py:method:: reset_grouping_ind() Reset the grouping step selection. .. py:method:: remove_and_reset_grouping() Re-initialize grouping and remove current data. .. py:method:: levels2data(use_grouped: bool = None, use_roi: bool = False, use_global_groups: bool = False) -> Tuple[numpy.ndarray, numpy.ndarray] Level plotting data. Uses the Particle objects' levels to generate two arrays for plotting the levels. :param use_grouped: Whether to use grouped levels - if not provided, defaults to True if groups exist and false if they don't. :type use_grouped: bool, optional :param use_global_groups: whether to use globally grouped levels :type use_global_groups: bool = False :param use_roi: whether to use ROI :type use_roi: bool = False :returns: **ints, times** -- Intensities as a function of time for plotting. :rtype: Tuple[np.ndarray, np.ndarray] .. py:method:: lifetimes2data(use_grouped: bool = None, use_roi: bool = False) -> Tuple[numpy.ndarray, numpy.ndarray] Level lifetime plotting data. Uses the Particle object's levels to generate two arrays for plotting the level lifetimes. :param use_grouped: Whether to use grouped levels - if not provided, defaults to True if groups exist and false if they don't. :type use_grouped: bool, optional :param use_roi: Whether to use ROI. :type use_roi: bool = False :returns: **lifetimes, times** -- Lifetime as a function of time for plotting. :rtype: Tuple[np.ndarray, np.ndarray] .. py:method:: current2data(level_ind: int, use_roi: bool = False) -> Tuple[numpy.ndarray, numpy.ndarray] Current level plotting data. Uses the Particle object's selected level to generate two arrays for plotting level. :param level_ind: Index of the level to be plotted. :type level_ind: int :param use_roi: Whether to use the ROI. :type use_roi: bool = False :returns: **levels_data, times** -- Intensity as a function of time for plotting. :rtype: Tuple[np.ndarray, np.ndarray] .. py:method:: current_group2data(group_ind: int) -> [numpy.ndarray, numpy.ndarray] Current group plotting data. Uses the Particle object's selected group to generate two arrays for plotting the group. :param group_ind: Index of the level to be plotted. :type group_ind: int :returns: **group_int, times** -- Intensity as a function of time for plotting. :rtype: Tuple[np.ndarray, np.ndarray] .. py:method:: makehistograms(remove_zeros, startpoint, channel) Make all histograms - whole trace and levels. :param remove_zeros: Whether to remove zeros at the start of the decay. :type remove_zeros: bool :param startpoint: Startpoint of the decay in number of time steps. :type startpoint: int :param channel: Whether to use the hardware channelwidth TODO: remove this parameter including downstream as it is never used :type channel: bool .. py:method:: makehistogram(channel=True, add_roi: bool = False) Put the arrival times into a histogram. :param channel: Whether to use the hardware channelwidth. :type channel: bool = True :param add_roi: Whether to create the ROI Histogram as well. :type add_roi: bool = True .. py:method:: makelevelhists(channel: bool = True, force_cpts_levels: bool = False, force_group_levels: bool = False) Make level histograms. :param channel: Whether to use the hardware channelwidth. :type channel: bool = True :param force_cpts_levels: Use self.cpts.levels instead of self.levels. :type force_cpts_levels: bool = False :param force_group_levels: Use self.group_levels instead of self.levels. :type force_group_levels: bool = False .. py:method:: makegrouplevelhists() Make grouped level histograms. .. py:method:: makegrouphists(channel=True) Make group histograms. .. py:method:: binints(binsize: int) Bin the absolute times into intensity trace. :param binsize: Size of intensity trace time bins. :type binsize: int .. py:method:: trim_trace(min_level_dwell_time: float, min_level_int: int, reset_roi: bool = True) Trim the intensity trace. This function trims the intensity trace to remove the photobleached end. :param min_level_dwell_time: Minimum dwell time of bleached level for trimming. :type min_level_dwell_time: float :param min_level_int: Minimum intensity to classify level as not bleached. :type min_level_int: int :param reset_roi: Whether to update the ROI to the trimmed area. :type reset_roi: bool = True .. py:class:: FakeCpts(num_levels: int, levels: list) Fake ChangePoints object for GlobalParticle. :param num_levels: number of intensity levels :type num_levels: int :param levels: list of global levels :type levels: List[GlobalLevel] .. py:attribute:: num_levels .. py:attribute:: levels .. py:attribute:: num_cpts .. py:attribute:: has_levels :value: 'True' .. py:class:: GlobalParticle(particles: List[Particle], use_roi: bool = False) Particle-like object containing levels from all particles. :param particles: Particles to include in global particle. :type particles: List[Particle] :param use_roi: Whether to use ROI's. :type use_roi: bool = False .. py:property:: has_levels .. py:property:: has_groups .. py:property:: groups .. py:property:: num_groups .. py:property:: groups_bounds .. py:property:: groups_ints .. py:property:: grouping_bics .. py:property:: grouping_selected_ind .. py:property:: best_grouping_ind .. py:property:: grouping_num_groups .. py:property:: level_particle_dataset_inds .. py:property:: global_levels :type: list .. py:attribute:: is_global :value: 'True' .. py:attribute:: name :value: "'Global Particle'" .. py:attribute:: contributing_particles_dataset_inds .. py:attribute:: levels :value: '[]' .. py:attribute:: num_levels :value: '0' .. py:attribute:: num_levels_roi :value: '0' .. py:attribute:: dwell_time .. py:attribute:: num_photons .. py:attribute:: uuid .. py:attribute:: use_roi_for_grouping :value: 'False' .. py:attribute:: cpts :value: 'None' .. py:attribute:: ahca .. py:method:: run_grouping() .. py:class:: Trace(particle: Particle, binsize: int) Binned intensity trace. :param particle: The Particle which creates the Trace. :type particle: Particle :param binsize: Size of time bin in ms. :type binsize: int .. py:attribute:: binsize .. py:attribute:: intdata .. py:attribute:: inttimes .. py:class:: Histogram(particle: Particle, level: Union[change_point.Level, List[int]] = None, start_point: float = None, channel: bool = True, trim_start: bool = False, is_for_roi: bool = False) TCSPC histogram. This class represents histogrammed TCSPC arrival times (micro times) as well as multi-exponential fits thereof. :param particle: The parent Particle of this object. :type particle: Particle :param level: The possible parent level of this object. :type level: Level or List = None :param start_point: Start point for lifetime fit. :type start_point: float = None :param channel: Whether to use hardware channel width. :type channel: bool = True :param trim_start: Whether to trim zeros at the start of the histogram. :type trim_start: bool = False :param is_for_roi: Whether this histogram is from a trace ROI. :type is_for_roi: bool = False .. py:property:: t .. py:attribute:: is_for_roi :value: 'False' .. py:attribute:: fitted_with_roi :value: 'None' .. py:attribute:: roi_region_used :value: 'None' .. py:attribute:: _particle .. py:attribute:: level :value: 'None' .. py:attribute:: original_kwargs .. py:attribute:: microtimes :value: 'None' .. py:attribute:: convd :value: 'None' .. py:attribute:: convd_t :value: 'None' .. py:attribute:: fitted :value: 'False' .. py:attribute:: fit_decay :value: 'None' .. py:attribute:: tau :value: 'None' .. py:attribute:: amp :value: 'None' .. py:attribute:: shift :value: 'None' .. py:attribute:: bg :value: 'None' .. py:attribute:: irfbg :value: 'None' .. py:attribute:: avtau :value: 'None' .. py:attribute:: numexp :value: 'None' .. py:attribute:: residuals :value: 'None' .. py:attribute:: fwhm :value: 'None' .. py:attribute:: stds :value: 'None' .. py:attribute:: avtaustd :value: 'None' .. py:attribute:: chisq :value: 'None' .. py:attribute:: dw :value: 'None' .. py:attribute:: dw_bound :value: 'None' .. py:attribute:: decay_roi_start_ns :value: 'None' .. py:attribute:: decay_roi_end_ns :value: 'None' .. py:attribute:: num_photons_used :value: 'None' .. py:method:: setup(level: Union[change_point.Level, List[int]] = None, start_point: float = None, channel: bool = True, trim_start: bool = False, use_roi: bool = False) Set up the object. This method can be called to re-do setup steps without reinitializing. :param level: The possible parent level of this object. :type level: Level or List = None :param start_point: Start point for lifetime fit. :type start_point: float = None :param channel: Whether to use hardware channel width. :type channel: bool = True :param trim_start: Whether to trim zeros at the start of the histogram. :type trim_start: bool = False :param use_roi: Whether this histogram is from a trace ROI. :type use_roi: bool = False .. py:method:: update_roi() Rerun setup to update ROI. .. py:method:: fit(numexp, tauparam, ampparam, shift, decaybg, irfbg, boundaries, addopt, irf, fwhm=None, normalize_amps=True, maximum_likelihood=False) Fit a multiexponential decay to the histogram. This function mainly calls the relevant code from `tcspcfit`. :param numexp: Number of exponentials in fit function (1-3). :type numexp: int :param tauparam: Initial guess times (in ns). This is either in the format [tau1, tau2, ...] or [[tau1, min1, max1, fix1], [tau2, ...], ...]. When the "fix" value is False, the min and max values are ignored. :type tauparam: array_like :param ampparam: Initial guess amplitude. Format [amp1, amp2, ...] or [[amp1, fix1], [amp2, fix2], ...] :type ampparam: array_like :param shift: Initial guess IRF shift. Either a float, or [shift, min, max, fix]. :type shift: array_like :param decaybg: Background value for decay. Will be estimated if not given. :type decaybg: float :param irfbg: Background value for IRF. Will be estimated if not given. :type irfbg: float :param boundaries: Start and end of fitting range as well as options for automatic determination of the parameters as used by `FluoFit.calculate_boundaries`. :type boundaries: list :param irf: Instrumental Response Function :type irf: ndarray :param fwhm: Full-width at half maximum of simulated irf. IRF is not simulated if fwhm is None. :type fwhm: float = None :param addopt: Additional options for `scipy.optimize.curve_fit` (such as optimization parameters). :type addopt: Dict = None :param normalize_amps: Whether to use normalized lifetime amplitudes. :type normalize_amps: bool = True :param maximum_likelihood: Whether to use maximum likelihood fitting (otherwise use least squares). :type maximum_likelihood: bool = False .. py:method:: levelhist(level) .. py:method:: start_at_value(decay, t, neg_t=True, decaystart=None) :staticmethod: Helper method for setting decay startpoint. .. py:class:: ParticleAllHists(particle: Particle) Class containing all Histograms from Particle. :param particle: The parent particle of this object. :type particle: Particle .. py:attribute:: part_uuid .. py:attribute:: numexp :value: 'None' .. py:attribute:: part_hist .. py:attribute:: has_level_hists .. py:attribute:: level_hists :value: '[]' .. py:attribute:: has_group_hists .. py:attribute:: group_hists :value: '[]' .. py:method:: fit_part_and_levels(channelwidth, start, end, fit_param: tcspcfit.FittingParameters) Fit all Histograms in this object. :param channelwidth: TCSPC channelwidth (time step size) in ns. :type channelwidth: float :param start: Fitting startpoint in number of time steps. :type start: int :param end: Fitting endpoint in number of time steps. :type end: int :param fit_param: Object containing fit parameters. :type fit_param: FittingParameters .. py:class:: RasterScan(h5dataset: H5dataset, particle_num: int, h5dataset_index: int, particle_indexes: List[int]) Class containing raster scan data. A raster scan is a 2D intensity scan used to visualize particles before measurement. :param h5dataset: The parent HDF5 dataset object. :type h5dataset: H5dataset :param particle_num: Number of particles connected to this raster scan. :type particle_num: int :param h5dataset_index: Index of this raster scan in the dataset. :type h5dataset_index: int :param particle_indexes: Dataset indices of the particles in this raster scan. :type particle_indexes: List[int] .. py:property:: dataset :type: h5pickle.Dataset .. py:attribute:: h5dataset .. py:attribute:: particle_num .. py:attribute:: h5dataset_index .. py:attribute:: particle_indexes .. py:attribute:: integration_time .. py:attribute:: pixel_per_line .. py:attribute:: range .. py:attribute:: x_start .. py:attribute:: y_start .. py:attribute:: x_axis_pos .. py:attribute:: y_axis_pos .. py:class:: Spectra(particle: Particle) Class containing spectral data. Spectra are recorded as a time scan, with a certain integration time over which a single spectrum is recorded using a grating and CCD. :param particle: The parent particle object. :type particle: Particle .. py:property:: _has_spectra :type: bool .. py:property:: data :type: h5pickle.Dataset .. py:property:: wavelengths :type: numpy.ndarray .. py:property:: series_times :type: numpy.ndarray .. py:attribute:: _particle