Utils API

cohere_core.config_verifier

Verification of configuration parameters.

cohere_core.config_verifier.verify(file_name, conf_map)

Verifies parameters.

Parameters:

  • file_name (str) – name of file the parameters are related to. Supported: config_prep, config_data, config_rec, config_disp

  • conf_map (dict) – parameters

Returns:

a message with description of error or empty string if no error

Return type:

str

cohere_core.utils

This module is a suite of utility functions.

cohere_core.utils.Resize(IN, dim)
Parameters:

  • IN

  • dim

Returns:

cohere_core.utils.adjust_dimensions(arr, pads)

This function adds to or subtracts from each dimension of the array elements defined by pad. If the pad is positive, the array is padded in this dimension. If the pad is negative, the array is cropped. The dimensions of the new array is then adjusted to be supported by the opencl library.

Parameters:

  • arr (ndarray) – the array to pad/crop

  • pad (list) – list of three pad values, for each dimension

Returns:

the padded/cropped array

Return type:

ndarray

cohere_core.utils.binning(array, binsizes)

This function does the binning of the array. The array is binned in each dimension by the corresponding binsizes elements. If binsizes list is shorter than the array dimensions, the remaining dimensions are not binned.

Parameters:

  • array (ndarray) – the original array to be binned

  • binsizes (list) – a list defining binning factors for corresponding dimensions

Returns:

binned array

Return type:

ndarray

cohere_core.utils.center_max(arr, center_shift)
This function finds maximum value in the array, and puts it in a center of a new array. If center_shift is not zeros,

the array will be shifted accordingly. The shifted elements are rolled into the other end of array.

Parameters:

  • arr (ndarray) – the original array to be centered

  • center_shift (list) – a list defining shift of the center

Returns:

centered array

Return type:

ndarray

cohere_core.utils.crop_center(arr, new_shape)

This function crops the array to the new size, leaving the array in the center. The new_size must be smaller or equal to the original size in each dimension. :param arr: the array to crop :type arr: ndarray :param new_shape: new size :type new_shape: tuple

Returns:

cropped – the cropped array

Return type:

ndarray

cohere_core.utils.estimate_no_proc(arr_size, factor)

Estimates number of processes the prep can be run on. Determined by number of available cpus and size of array. :param arr_size: size of array :type arr_size: int :param factor: an estimate of how much memory is required to process comparing to array size :type factor: int

Returns:

number of processes

Return type:

int

cohere_core.utils.get_avail_gpu_runs(devices, run_mem)

Finds how many jobs of run_mem size can run on configured GPUs on local host.

Parameters:

  • devices – list or string list of GPU IDs or ‘all’ if configured to use all available GPUs

  • run_mem – int size of GPU memory (in MB) needed for one job

Returns:

dict pairs of GPU IDs, number of available jobs

cohere_core.utils.get_central_object_extent(fp: numpy.ndarray) → list
Parameters:

fp

Returns:

cohere_core.utils.get_good_dim(dim)

This function calculates the dimension supported by opencl library (i.e. is multiplier of 2, 3, or 5) and is closest to the given starting dimension. If the dimension is not supported the function adds 1 and verifies the new dimension. It iterates until it finds supported value.

Parameters:

dim (int) – initial dimension

Returns:

a dimension that is supported by the opencl library, and closest to the original dimension

Return type:

int

cohere_core.utils.get_gpu_use(devices, no_jobs, job_size)

Determines available GPUs that match configured devices, and selects the optimal distribution of jobs on available devices. If devices is configured as dict (i.e. cluster configuration) then a file “hosts” is created in the running directory. This file contains hosts names and number of jobs to run on that host. :param devices: Configured parameter. list of GPU ids to use for jobs or ‘all’ if all GPUs should be used. If cluster configuration, then

it is dict with keys being host names.

Parameters:

  • no_jobs (int) – wanted number of jobs

  • job_size (float) – a GPU memory requirement to run one job

Returns:

  • picked_devs (list or list of lists(if cluster conf)) – list of GPU ids that were selected for the jobs

  • available jobs (int) – number of jobs allocated on all GPUs

  • cluster_conf (boolean) – True is cluster configuration

cohere_core.utils.get_logger(name, ldir='')

Creates looger instance that will write to default.log file in a given directory. :param name: logger name :type name: str :param ldir: directory where to create log file :type ldir: str

Returns:

logger object from logging module

Return type:

logger

cohere_core.utils.get_one_dev(ids)

Returns GPU ID that is included in the configuration, is on a local node, and has the most available memory.

Parameters:

ids – list or string or dict list of gpu ids, or string ‘all’ indicating all GPUs included, or dict by hostname

Returns:

int selected GPU ID

cohere_core.utils.get_oversample_ratio(fp: numpy.ndarray) → numpy.ndarray
Parameters:

fp – diffraction pattern

Returns:

oversample ratio in each dimension

cohere_core.utils.pad_center(arr, new_shape)

This function pads the array with zeros to the new shape with the array in the center. :param arr: the original array to be padded :type arr: ndarray :param new_shape: new dimensions :type new_shape: tuple

Returns:

the zero padded centered array

Return type:

ndarray

cohere_core.utils.read_config(config)

This function gets configuration file. It checks if the file exists and parses it into a dictionary.

Parameters:

config (str) – configuration file name

Returns:

dictionary containing parsed configuration, None if the given file does not exist

Return type:

dict

cohere_core.utils.read_results(read_dir)

Reads results and returns array representation.

Parameters:

read_dir (str) – directory to read the results from

Returns:

image, support, and coherence arrays

Return type:

ndarray, ndarray, ndarray (or None)

cohere_core.utils.read_tif(filename)

This method reads tif type file and returns the data as ndarray.

Parameters:

filename (str) – tif format file name

Returns:

an array containing the data parsed from the file

Return type:

ndarray

cohere_core.utils.save_metrics(errs, dir, metrics=None)

Saves arrays metrics and errors by iterations in text file.

Parameters:

  • errs (list) – list of “chi” error by iteration

  • dir (str) – directory to write the file containing array metrics

  • metrics (dict) – dictionary with metric type keys, and metric values

cohere_core.utils.save_results(image, support, coh, errs, save_dir, metric=None)

Saves results of reconstruction. Saves the following files: image.np, support.npy, errors.npy, optionally coherence.npy, plot_errors.py, graph.npy, flow.npy, iter_array.npy

Parameters:

  • image (ndarray) – reconstructed image array

  • support (ndarray) – support array related to the image

  • coh (ndarray) – coherence array when pcdi feature is active, None otherwise

  • errs (ndarray) – errors “chi” by iterations

  • save_dir (str) – directory to write the files

  • metrics (dict) – dictionary with metric type keys, and metric values

cohere_core.utils.save_tif(arr, filename)

This method saves array in tif format file.

Parameters:

  • arr (ndarray) – array to save

  • filename (str) – tif format file name

cohere_core.utils.select_central_object(fp: numpy.ndarray) → numpy.ndarray
Parameters:

fp

Returns:

cohere_core.utils.threshold_by_edge(fp: numpy.ndarray) → numpy.ndarray

Author: Yudong Yao

Parameters:

fp

Returns:

cohere_core.utils.write_config(param_dict, config)

Writes configuration to a file. :param param_dict: dictionary containing configuration parameters :type param_dict: dict :param config: configuration name theparameters will be written into :type config: str

cohere_core.utils.write_plot_errors(save_dir)

Creates python executable that draw plot of error by iteration. It assumes that the given directory contains “errors.npy” file

Parameters:

save_dir (str) – directory containing errors.npy file