Functionality

ClearMap 2.0 is designed for advanced and fast image processing of large (Terabyte) 3D data sets [Kirst2020] obtained from tissue clearing. It is a complete redesign of ClearMap 1.0 [Renier2016].

The modular source management allows a unified handling of various data sources (image files, binary files, memory maps, shared memory arrays, numpy arrays, GPU arrays, graph formats) and fast parallel 3D image and graph processing as well as interactive visualization of large 3d images.

In the following we provide a list of the ClearMap 2.0 functionality. ClearMap 2.0 is open source software and available for download under https://github.com/ChristophKirst/ClearMap2.

Modules

The ClearMap code is structured into these main modules:

• Alignment for stitching, resampling, and registration of images onto references

• Analysis for measurements and statistical analysis of the data

• ImageProcessing for correcting and quantifying image data

• IO for reading and writing data

• ParallelProcessing for organizing parallel processing of the data

• Visualization for visualizing the data and results.

Alignment

The ClearMap.Alignment module includes methods for

• 3d resampling:

  • Resampling

• 3d alignment to reference atlases (e.g. Allen Brain Institute Atlases) via interfacing to elastix:

  • Elastix

  • Annotation

• 3d rigid and wobbly stitching via WobblyStitcher:

  • StitchingRigid

  • StitchingWobbly

Analysis

The Analysis module provides methods for the quantification and analysis of the data, including:

• graph and network analysis for very large graphs

  • Graphs

  enabling:

  • graph preprocessing and cleanup

  • graph branch reduction

  • graph annotation

  • graphs embedded 3d space

  • 3d graphs with 3d edge geometry

  • graph vertex and edge label morphological operations

  • sub-graph extraction

  • network analysis

• measurement routines to quantify the results

  • Measurements

  enabling:

  • measurments of expression levels

  • measurments of geometric shapes and radii

  • maxima detection

  • voxelization

• statistics routines to analyse the results

  • Statistics

ImageProcessing

The ImageProcessing module is the core of ClearMap and provides methods to process 3d images of TB size, including:

• binary image processing

  • Binary

  with filling and discrete topology based binary smoothing routines:

  • Filling

  • Smoothing

• clipping and normalization

  • Clipping

• 3d local gradients and Hessian matrices

  • Differentiation

  with 3d tube filter and tubeness measures:

  • Hessian

• 3d filtering

  • Filter

  with 3d rank filter library (>30 filters)

  • Rank

• skeletonization via parallel thinning

  • Skeletonization

• 3d tracing

  • Tracing

• fast calculation of 3d local image statistics

  • LocalStatistics

• fast 3d local image statistics

  • LocalStatistics

• hysteresis and seeded thresholding

  • Thresholding

• light-sheet artifact removal

  • LightsheetCorrection

• illuminatoin correction

  • IlluminationCorrection

• machine learning based image processing

  • MachineLearning

  with vessel and tube filling deep convolutional neural network:

  • VesselFilling

• expert processing pipelines for specific applications

  • Experts

IO

The IO module provides methods to hold information about data and read and write data files efficiently as sources and sinks, that can be sliced and turned into virtual sources useful for parallel processing and memory mapping:

• Source

• Slice

Supported source types include:

Format	Module	Description
tif	TIF	tif images and stacks
raw / mhd	MHD	raw image files with optional mhd header file
nrrd	NRRD	nearly raw raster data files
csv	CSV	text files as comma separated values
npy	NPY	numpy binary file
gt	GT	graph tool file
file list	FileList	folder, file list or file expression for a list source files
	MMP	memory mapped file
	SMA	shared memory array

The IO module also provides a workspace that handles the organization of files in CellMap and TubeMap projects:

• Workspace

ParallelProcessing

The ParallelProcessing module provides methods for distributed processing.

• large data arrays can be proccessed in blocks via a specialized IO source

  • Block

  • BlockProcessing

• or via shared memory arrays:

  • SharedMemoryArray

  • SharedMemoryManager

• numerical processing routines for TB data arrays are collected in

  • DataProcessing

  including speed ups for numpy array processing, convolution and measurement routines:

  • ArrayProcessing

  • ConvolvePointList

  • DevolvePointList

  • MeasurePointList

  • StatisticsPointList

Visualization

ClearMap’s “mod:ClearMap.Visualization module comes with a larger set of interactive visualization functions using various backends:

• Qt

  providing:

  • fast interactive 2d slice plotting of 3d TB data sets

    • DataViewer

  • overlays and/or synchronized window display of multiple data sets.

    • Plot3d

• Vispy

  providing:

  • 3d volume rendering, lists and line plots

    • Plot3d

  • 3d plots 3d graphs as lines or mesh plots with edge geometries, etc

    • PlotGraph3d