IO

IO interface to read files as sources.

This is the main module to distribute the reading and writing of individual data formats to the specialized sub-modules.

See ClearMap.IO for details.

as_source(source, slicing=None, *args, **kwargs)

Convert source specification to a Source class.

Arguments

sourceobject

The source specification.

Returns

sourceSource class

The source class.

buffer(source)

Returns an io buffer of the data array of a source for use with e,g python.

Arguments

sourcesource specification

The source specification.

Returns

bufferarray or memmap

A buffer to read and write data.

convert(source, sink, processes=None, verbose=False, **kwargs)

Transforms a source into another format.

Arguments

sourcesource specification

The source or list of sources.

sinksource specification

The sink or list of sinks.

Returns

sinksink speicication

The sink or list of sinkfs.

convert_files(filenames, extension=None, path=None, processes=None, verbose=False)

Transforms list of files to their sink format in parallel.

Arguments

filenameslist of str

The filenames to convert

extensionstr

The new file format extension.

pathstr or None

Optional path speicfication.

processesint, ‘serial’ or None

The number of processes to use for parallel conversion.

verbosebool

If True, print progress information.

Returns

filenameslist of str

The new file names.

create(source, *args, **kwargs)

Create a data source on disk.

Arguments

sourcestr, array, Source class

The source to write data to.

Returns

sinkstr, array or Source class

The sink to which the data was written.

dtype(source)

Returns dtype of a source.

Arguments

sourcestr, array or Source

The source specification.

Returns

dtypedtype

The data type of the source.

element_strides(source)

Returns the strides of the data array of a source.

Arguments

sourcestr, array, dtype or Source

The source specification.

Returns

stridestuple of int

The strides of the souce.

file_list(expression=None, file_list=None, sort=True, verbose=False)

Returns the list of files that match the tag expression.

Arguments

expression :str

The regular expression the file names should match.

sortbool

If True, sort files naturally.

verbosebool

If True, print warning if no files exists.

Returns

file_listlist of str

The list of files that matched the expression.

filename_to_module(filename)

Returns the IO module associated with a filename.

Arguments

filenamestr

The file name.

Returns

modulemodule

The module that handles the IO of the file.

initialize(source=None, shape=None, dtype=None, order=None, location=None, memory=None, like=None, hint=None, **kwargs)

Initialize a source with specified properties.

Arguments

sourcestr, array, Source class

The source to write data to.

shapetuple or None

The desired shape of the source. If None, infered from existing file or from the like parameter. If not None and source has a valid shape shapes are tested to match.

dtypetype, str or None

The desired dtype of the source. If None, infered from existing file or from the like parameter. If not None and source has a valid dtype the types are tested to match.

order‘C’, ‘F’ or None

The desired order of the source. If None, infered from existing file or from the like parameter. If not None and source has a valid order the orders are tested to match.

locationstr or None

The desired location of the source. If None, infered from existing file or from the like parameter. If not None and source has a valid location the locations need to match.

memory‘shared’ or None

The memory type of the source. If ‘shared’ a shared array is created.

likestr, array or Source class

Infer the source parameter from this source.

hintstr, array or Source class

If parameters for source creation are missing use the ones from this hint source.

Returns

sourceSource class

The initialized source.

Note

The source is created on disk or in memory if it does not exists so processes can start writing into it.

initialize_buffer(source, shape=None, dtype=None, order=None, location=None, memory=None, like=None, **kwargs)

Initialize a buffer with specific properties.

Arguments

sourcestr, array, Source class

The source to write data to.

shapetuple or None

The desired shape of the source. If None, infered from existing file or from the like parameter. If not None and source has a valid shape shapes are tested to match.

dtypetype, str or None

The desired dtype of the source. If None, infered from existing file or from the like parameter. If not None and source has a valid dtype the types are tested to match.

order‘C’, ‘F’ or None

The desired order of the source. If None, infered from existing file or from the like parameter. If not None and source has a valid order the orders are tested to match.

locationstr or None

The desired location of the source. If None, infered from existing file or from the like parameter. If not None and source has a valid location the locations need to match.

memory‘shared’ or None

The memory type of the source. If ‘shared’ a shared array is created.

likestr, array or Source class

Infer the source parameter from this source.

Returns

bufferarray

The initialized buffer to use tih e.g. cython.

Note

The buffer is created if it does not exists.

is_source(source, exists=True)

Checks if source is a valid source.

Arguments

sourceobject

Source to check.

existsbool

If True, check if source exists in case it has a location.

Returns

is_sourcebool

True if source is a valid source.

location(source)

Returns the location of a source.

Arguments

sourcestr, array or Source

The source specification.

Returns

locationstr or None

The location of the source.

location_to_module(location)

Returns the IO module associated with a location string.

Arguments

locationobject

Location of the source.

Returns

modulemodule

The module that handles the IO of the source specified by its location.

max_value(source)

Returns the maximal value of the data type of a source.

Arguments

sourcestr, array, dtype or Source

The source specification.

Returns

max_valuenumber

The maximal value for the data type of the source

memory(source)

Returns the memory type of a source.

Arguments

sourcestr, array or Source

The source specification.

Returns

memorystr or None

The memory type of the source.

min_value(source)

Returns the minimal value of the data type of a source.

Arguments

sourcestr, array, dtype or Source

The source specification.

Returns

min_valuenumber

The minimal value for the data type of the source

ndim(source)

Returns number of dimensions of a source.

Arguments

sourcestr, array or Source

The source specification.

Returns

ndimint

The number of dimensions in the source.

order(source)

Returns order of a source.

Arguments

sourcestr, array or Source

The source specification.

Returns

order‘C’, ‘F’, or None

The order of the source data items.

read(source, *args, **kwargs)

Read data from a data source.

Arguments

sourcestr, array, Source class

The source to read the data from.

Returns

dataarray

The data of the source.

shape(source)

Returns shape of a source.

Arguments

sourcestr, array or Source

The source specification.

Returns

shapetuple of ints

The shape of the source.

size(source)

Returns size of a source.

Arguments

sourcestr, array or Source

The source specification.

Returns

sizeint

The size of the source.

source(source, slicing=None, *args, **kwargs)

Convert source specification to a Source class.

Arguments

sourceobject

The source specification.

Returns

sourceSource class

The source class.

source_to_module(source)

Returns IO module associated with a source.

Arguments

sourceobject

The source specification.

Returns

typemodule

The module that handles the IO of the source.

write(sink, data, *args, **kwargs)

Write data to a data source.

Arguments

sinkstr, array, Source class

The source to write data to.

dataarray

The data to write to the sink.

slicingslice specification or None

Optional subslice to write data to.

Returns

sinkstr, array or Source class

The sink to which the data was written.

file_extension_to_module = {'csv': <module 'ClearMap.IO.CSV' from '/home/ckirst/Programs/ClearMap2/ClearMap/IO/CSV.py'>, 'gt': <module 'ClearMap.IO.GT' from '/home/ckirst/Programs/ClearMap2/ClearMap/IO/GT.py'>, 'npy': <module 'ClearMap.IO.MMP' from '/home/ckirst/Programs/ClearMap2/ClearMap/IO/MMP.py'>, 'nrdh': <module 'ClearMap.IO.NRRD' from '/home/ckirst/Programs/ClearMap2/ClearMap/IO/NRRD.py'>, 'nrrd': <module 'ClearMap.IO.NRRD' from '/home/ckirst/Programs/ClearMap2/ClearMap/IO/NRRD.py'>, 'tif': <module 'ClearMap.IO.TIF' from '/home/ckirst/Programs/ClearMap2/ClearMap/IO/TIF.py'>}

Map between file extensions and modules that handle this file type.

source_modules = [<module 'ClearMap.IO.NPY' from '/home/ckirst/Programs/ClearMap2/ClearMap/IO/NPY.py'>, <module 'ClearMap.IO.TIF' from '/home/ckirst/Programs/ClearMap2/ClearMap/IO/TIF.py'>, <module 'ClearMap.IO.MMP' from '/home/ckirst/Programs/ClearMap2/ClearMap/IO/MMP.py'>, <module 'ClearMap.IO.SMA' from '/home/ckirst/Programs/ClearMap2/ClearMap/IO/SMA.py'>, <module 'ClearMap.IO.FileList' from '/home/ckirst/Programs/ClearMap2/ClearMap/IO/FileList.py'>, <module 'ClearMap.IO.NRRD' from '/home/ckirst/Programs/ClearMap2/ClearMap/IO/NRRD.py'>, <module 'ClearMap.IO.CSV' from '/home/ckirst/Programs/ClearMap2/ClearMap/IO/CSV.py'>, <module 'ClearMap.IO.GT' from '/home/ckirst/Programs/ClearMap2/ClearMap/IO/GT.py'>]

The valid source modules.