How to extract features from fMOST data with neuron_morphology

To obtain the package, please see https://neuron-morphology.readthedocs.io/en/readthedocs/.

This step-by-step guide will walk you through the process of extracting features from fMOST data.

• Load fMOST data from file
• Clean up swc to prevent errors
• Extract the neuron_morphology default_features
• Add a custom feature to calculate as well

import sys
sys.path.insert(0, "../")

from io import StringIO
import copy
import matplotlib.pyplot as plt

import neuron_morphology.swc_io as swcio
from neuron_morphology.morphology import Morphology
from neuron_morphology.swc_io import morphology_from_swc
from neuron_morphology.feature_extractor.data import Data
from neuron_morphology.feature_extractor.feature_extractor import FeatureExtractor
from neuron_morphology.features.default_features import default_features
from neuron_morphology.constants import (
    SOMA, AXON, BASAL_DENDRITE, APICAL_DENDRITE
)

import json
import numpy as np
import neuron_morphology.feature_extractor.feature_writer as fw

Pick your neuron (.swc/.csv)

Here select one SWC file as an example.

fmost_swc_file = "../../tests/data/17545-6151-X24259-Y36270.swc"

Ensure one SWC with only one soma point

This is developed for current registered fMOST SWC file. It will adjusted and added as a module into neuron_morphology later.

def prepare_neuron_tree(swc_data):
    nodes = swc_data.to_dict('record')

    replace_type = 2 # default node type
    for node in nodes:
        node['parent'] = int(node['parent'])
        node['id'] = int(node['id'])
        node['type'] = int(node['type'])

        if node['parent'] == -1 and node['type'] != 1:
            replace_type = node['type']

        if node['type'] == 1 and node['parent'] != -1:
            node['type'] = replace_type

    soma_list = []
    for node in nodes:
        if node['type'] == 1:
            soma_list.append(node)

    # create a new soma point
    if len(soma_list) > 1:
        x = 0
        y = 0
        z = 0
        n = len(soma_list)
        for node in soma_list:
            x += node['x']
            y += node['y']
            z += node['z']

        soma = copy.deepcopy(soma_list[0])
        soma['id'] = nodes[-1]['id']
        soma['x'] = x/n
        soma['y'] = y/n
        soma['z'] = z/n
        nodes.append(soma)

        for node in soma_list:
            node['parent'] = soma['id']
            node['type'] = replace_type
    
    return nodes

Load SWC file as morphology data

We can load the SWC file into our morphology data object and then calculate features on it.

# load
swc_data = swcio.read_swc(fmost_swc_file)

nodes = prepare_neuron_tree(swc_data)
test_data = Data(Morphology(nodes, node_id_cb=lambda node: node['id'], parent_id_cb=lambda node: node['parent']))

# visualize
x = [node['x'] for node in nodes]
y = [node['y'] for node in nodes]
z = [node['z'] for node in nodes]

fig, ax = plt.subplots(1, 2)
ax[0].scatter(x, y, s=0.1)
ax[0].set_title('x-y view')
ax[1].scatter(z, y, s=0.1)
ax[1].set_title('z-y view')

Text(0.5, 1.0, 'z-y view')

Define interested features set

First, we instantiate a FeatureExtractor Second, we will use register the default_features

fe = FeatureExtractor()
fe.register_features(default_features)

<neuron_morphology.feature_extractor.feature_extractor.FeatureExtractor at 0x1a259f70f0>

Then, let's add a custom feature. We will use a feature included in the neurom_morphology package which calculates the number of stems exiting from the soma, by neurite type.

For reference, here is the entire function:

@marked(RequiresSoma)  
@marked(RequiresRoot) 
def calculate_number_of_stems(data: Data, node_types: Optional[List[int]]):

    """
        Calculate the number of soma stems.
        This is defined as the total number of non-soma child nodes on soma nodes.

        Parameters
        ----------
        data: Data Object containing a morphology

        node_types: list (AXON, BASAL_DENDRITE, APICAL_DENDRITE)
        Type to restrict search to

        Returns
        -------

        Scalar value

    """

    soma = data.morphology.get_soma()
    return len(data.morphology.children_of(soma))

This function has been marked to indicate that it needs a soma in order to be valid, and is also specializable based on node types. We will use the specialize() function and the NEURITE_SPECIALIZATION to tell the feature extractor to calculate this feature for each neurite_type (AXON, DENDRITE, BASAL_DENDRITE, APICAL_DENDRITE, ALL_NEURITES)

from neuron_morphology.features.soma import calculate_number_of_stems

from neuron_morphology.feature_extractor.marked_feature import specialize
from neuron_morphology.feature_extractor.feature_specialization import NEURITE_SPECIALIZATIONS

fe.register_features([specialize(calculate_number_of_stems, NEURITE_SPECIALIZATIONS)])

<neuron_morphology.feature_extractor.feature_extractor.FeatureExtractor at 0x1a259f70f0>

Extracting features

Now that we have registered the features that we are interested in, we can call feature extractor on our swc. This will create a dictionary of results, which we can unnest and print below.

Calling fe.extract() on another test_data will calculate the same feature set, so it can be used to process many swcs.

feature_extraction_run = fe.extract(test_data)
results = feature_extraction_run.results
from neuron_morphology.feature_extractor.utilities import unnest
unnest(results)

2020-02-25 13:13:47,771 root         INFO     skipping mark (validation failed): type
2020-02-25 13:13:47,773 root         INFO     skipping mark (validation failed): type
2020-02-25 13:13:47,774 root         INFO     skipping mark (validation failed): type
/Users/matthew.aitken/allen_dev/neuron_morphology/neuron_morphology/feature_extractor/mark.py:118: UserWarning: This morphology is not uniquely rooted! Found 279 root nodes. Features using the root node of this morphology may not select that node consistently. Some or all of these root nodes may not be soma nodes.
  f"This morphology is not uniquely rooted! Found {num_roots} "
2020-02-25 13:13:47,780 root         INFO     selected marks: <bound method FeatureExtractionRun.select_marks of <neuron_morphology.feature_extractor.feature_extraction_run.FeatureExtractionRun object at 0x1a25a2b358>>
2020-02-25 13:13:47,780 root         INFO     skipping feature: apical_dendrite.node.dimension. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,781 root         INFO     skipping feature: apical_dendrite.tip.dimension. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,782 root         INFO     skipping feature: apical_dendrite.compartment.dimension. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,783 root         INFO     skipping feature: apical_dendrite.bifurcation.dimension. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,785 root         INFO     skipping feature: apical_dendrite.num_nodes. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,787 root         INFO     skipping feature: apical_dendrite.num_branches. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,788 root         INFO     skipping feature: apical_dendrite.num_tips. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,789 root         INFO     skipping feature: apical_dendrite.mean_fragmentation. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,789 root         INFO     skipping feature: apical_dendrite.max_branch_order. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,791 root         INFO     skipping feature: apical_dendrite.num_outer_bifurcations. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,791 root         INFO     skipping feature: apical_dendrite.total_length. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,792 root         INFO     skipping feature: apical_dendrite.total_surface_area. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,793 root         INFO     skipping feature: apical_dendrite.total_volume. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,794 root         INFO     skipping feature: apical_dendrite.mean_diameter. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,795 root         INFO     skipping feature: apical_dendrite.mean_parent_daughter_ratio. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,795 root         INFO     skipping feature: apical_dendrite.max_euclidean_distance. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,796 root         INFO     skipping feature: apical_dendrite.dendrite.overlap. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,796 root         INFO     skipping feature: apical_dendrite.axon.overlap. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,798 root         INFO     skipping feature: basal_dendrite.apical_dendrite.overlap. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,798 root         INFO     skipping feature: dendrite.apical_dendrite.overlap. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,799 root         INFO     skipping feature: axon.apical_dendrite.overlap. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,800 root         INFO     skipping feature: apical_dendrite.apical_dendrite.overlap. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,801 root         INFO     skipping feature: apical_dendrite.basal_dendrite.overlap. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,802 root         INFO     skipping feature: apical_dendrite.node.moments. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,802 root         INFO     skipping feature: apical_dendrite.tip.moments. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,804 root         INFO     skipping feature: apical_dendrite.compartment.moments. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,804 root         INFO     skipping feature: apical_dendrite.bifurcation.moments. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,805 root         INFO     skipping feature: apical_dendrite.normalized_depth_histogram. Found extra marks: ['RequiresReferenceLayerDepths', 'RequiresLayeredPointDepths', 'RequiresApical']
2020-02-25 13:13:47,806 root         INFO     skipping feature: axon.normalized_depth_histogram. Found extra marks: ['RequiresReferenceLayerDepths', 'RequiresLayeredPointDepths']
2020-02-25 13:13:47,807 root         INFO     skipping feature: all_neurites.normalized_depth_histogram. Found extra marks: ['RequiresReferenceLayerDepths', 'RequiresLayeredPointDepths']
2020-02-25 13:13:47,807 root         INFO     skipping feature: dendrite.normalized_depth_histogram. Found extra marks: ['RequiresReferenceLayerDepths', 'RequiresLayeredPointDepths']
2020-02-25 13:13:47,808 root         INFO     skipping feature: basal_dendrite.normalized_depth_histogram. Found extra marks: ['RequiresReferenceLayerDepths', 'RequiresLayeredPointDepths']
2020-02-25 13:13:47,809 root         INFO     skipping feature: basal_dendrite.dendrite.earth_movers_distance. Found extra marks: ['RequiresReferenceLayerDepths', 'RequiresLayeredPointDepths']
2020-02-25 13:13:47,810 root         INFO     skipping feature: dendrite.dendrite.earth_movers_distance. Found extra marks: ['RequiresReferenceLayerDepths', 'RequiresLayeredPointDepths']
2020-02-25 13:13:47,811 root         INFO     skipping feature: axon.dendrite.earth_movers_distance. Found extra marks: ['RequiresReferenceLayerDepths', 'RequiresLayeredPointDepths']
2020-02-25 13:13:47,812 root         INFO     skipping feature: apical_dendrite.dendrite.earth_movers_distance. Found extra marks: ['RequiresReferenceLayerDepths', 'RequiresLayeredPointDepths', 'RequiresApical']
2020-02-25 13:13:47,813 root         INFO     skipping feature: basal_dendrite.axon.earth_movers_distance. Found extra marks: ['RequiresReferenceLayerDepths', 'RequiresLayeredPointDepths']
2020-02-25 13:13:47,814 root         INFO     skipping feature: dendrite.axon.earth_movers_distance. Found extra marks: ['RequiresReferenceLayerDepths', 'RequiresLayeredPointDepths']
2020-02-25 13:13:47,814 root         INFO     skipping feature: axon.axon.earth_movers_distance. Found extra marks: ['RequiresReferenceLayerDepths', 'RequiresLayeredPointDepths']
2020-02-25 13:13:47,815 root         INFO     skipping feature: apical_dendrite.axon.earth_movers_distance. Found extra marks: ['RequiresReferenceLayerDepths', 'RequiresLayeredPointDepths', 'RequiresApical']
2020-02-25 13:13:47,816 root         INFO     skipping feature: basal_dendrite.apical_dendrite.earth_movers_distance. Found extra marks: ['RequiresReferenceLayerDepths', 'RequiresLayeredPointDepths', 'RequiresApical']
2020-02-25 13:13:47,817 root         INFO     skipping feature: dendrite.apical_dendrite.earth_movers_distance. Found extra marks: ['RequiresReferenceLayerDepths', 'RequiresLayeredPointDepths', 'RequiresApical']
2020-02-25 13:13:47,818 root         INFO     skipping feature: axon.apical_dendrite.earth_movers_distance. Found extra marks: ['RequiresReferenceLayerDepths', 'RequiresLayeredPointDepths', 'RequiresApical']
2020-02-25 13:13:47,819 root         INFO     skipping feature: apical_dendrite.apical_dendrite.earth_movers_distance. Found extra marks: ['RequiresReferenceLayerDepths', 'RequiresLayeredPointDepths', 'RequiresApical']
2020-02-25 13:13:47,820 root         INFO     skipping feature: basal_dendrite.basal_dendrite.earth_movers_distance. Found extra marks: ['RequiresReferenceLayerDepths', 'RequiresLayeredPointDepths']
2020-02-25 13:13:47,822 root         INFO     skipping feature: dendrite.basal_dendrite.earth_movers_distance. Found extra marks: ['RequiresReferenceLayerDepths', 'RequiresLayeredPointDepths']
2020-02-25 13:13:47,824 root         INFO     skipping feature: axon.basal_dendrite.earth_movers_distance. Found extra marks: ['RequiresReferenceLayerDepths', 'RequiresLayeredPointDepths']
2020-02-25 13:13:47,825 root         INFO     skipping feature: apical_dendrite.basal_dendrite.earth_movers_distance. Found extra marks: ['RequiresReferenceLayerDepths', 'RequiresLayeredPointDepths', 'RequiresApical']
2020-02-25 13:13:47,826 root         INFO     skipping feature: apical_dendrite.calculate_number_of_stems. Found extra marks: ['RequiresApical']
2020-02-25 13:13:47,827 root         INFO     selected features: ['axon.node.dimension', 'all_neurites.node.dimension', 'dendrite.node.dimension', 'basal_dendrite.node.dimension', 'axon.tip.dimension', 'all_neurites.tip.dimension', 'dendrite.tip.dimension', 'basal_dendrite.tip.dimension', 'axon.compartment.dimension', 'all_neurites.compartment.dimension', 'dendrite.compartment.dimension', 'basal_dendrite.compartment.dimension', 'axon.bifurcation.dimension', 'all_neurites.bifurcation.dimension', 'dendrite.bifurcation.dimension', 'basal_dendrite.bifurcation.dimension', 'axon.num_nodes', 'all_neurites.num_nodes', 'dendrite.num_nodes', 'basal_dendrite.num_nodes', 'axon.num_branches', 'all_neurites.num_branches', 'dendrite.num_branches', 'basal_dendrite.num_branches', 'axon.num_tips', 'all_neurites.num_tips', 'dendrite.num_tips', 'basal_dendrite.num_tips', 'axon.mean_fragmentation', 'all_neurites.mean_fragmentation', 'dendrite.mean_fragmentation', 'basal_dendrite.mean_fragmentation', 'axon.max_branch_order', 'all_neurites.max_branch_order', 'dendrite.max_branch_order', 'basal_dendrite.max_branch_order', 'axon.num_outer_bifurcations', 'all_neurites.num_outer_bifurcations', 'dendrite.num_outer_bifurcations', 'basal_dendrite.num_outer_bifurcations', 'axon.total_length', 'all_neurites.total_length', 'dendrite.total_length', 'basal_dendrite.total_length', 'axon.total_surface_area', 'all_neurites.total_surface_area', 'dendrite.total_surface_area', 'basal_dendrite.total_surface_area', 'axon.total_volume', 'all_neurites.total_volume', 'dendrite.total_volume', 'basal_dendrite.total_volume', 'axon.mean_diameter', 'all_neurites.mean_diameter', 'dendrite.mean_diameter', 'basal_dendrite.mean_diameter', 'axon.mean_parent_daughter_ratio', 'all_neurites.mean_parent_daughter_ratio', 'dendrite.mean_parent_daughter_ratio', 'basal_dendrite.mean_parent_daughter_ratio', 'axon.max_euclidean_distance', 'all_neurites.max_euclidean_distance', 'dendrite.max_euclidean_distance', 'basal_dendrite.max_euclidean_distance', 'max_path_distance', 'early_branch_path', 'mean_contraction', 'basal_dendrite.dendrite.overlap', 'dendrite.dendrite.overlap', 'axon.dendrite.overlap', 'basal_dendrite.axon.overlap', 'dendrite.axon.overlap', 'axon.axon.overlap', 'basal_dendrite.basal_dendrite.overlap', 'dendrite.basal_dendrite.overlap', 'axon.basal_dendrite.overlap', 'axon.node.moments', 'all_neurites.node.moments', 'dendrite.node.moments', 'basal_dendrite.node.moments', 'axon.tip.moments', 'all_neurites.tip.moments', 'dendrite.tip.moments', 'basal_dendrite.tip.moments', 'axon.compartment.moments', 'all_neurites.compartment.moments', 'dendrite.compartment.moments', 'basal_dendrite.compartment.moments', 'axon.bifurcation.moments', 'all_neurites.bifurcation.moments', 'dendrite.bifurcation.moments', 'basal_dendrite.bifurcation.moments', 'axon.calculate_number_of_stems', 'all_neurites.calculate_number_of_stems', 'dendrite.calculate_number_of_stems', 'basal_dendrite.calculate_number_of_stems']

{'axon.node.dimension.width': 1967.175000000001,
 'axon.node.dimension.height': 3112.875,
 'axon.node.dimension.depth': 1416.8250000000003,
 'axon.node.dimension.min_xyz': array([ -194.125, -2929.8  , -1205.35 ]),
 'axon.node.dimension.max_xyz': array([1773.05 ,  183.075,  211.475]),
 'axon.node.dimension.bias_xyz': array([1578.925, 2746.725,  993.875]),
 'all_neurites.node.dimension.width': 1967.175000000001,
 'all_neurites.node.dimension.height': 3112.875,
 'all_neurites.node.dimension.depth': 1416.8250000000003,
 'all_neurites.node.dimension.min_xyz': array([ -194.125, -2929.8  , -1205.35 ]),
 'all_neurites.node.dimension.max_xyz': array([1773.05 ,  183.075,  211.475]),
 'all_neurites.node.dimension.bias_xyz': array([1578.925, 2746.725,  993.875]),
 'dendrite.node.dimension.width': 222.4250000000011,
 'dendrite.node.dimension.height': 201.60000000000127,
 'dendrite.node.dimension.depth': 196.77500000000055,
 'dendrite.node.dimension.min_xyz': array([ 967.7  , -284.375,  -72.85 ]),
 'dendrite.node.dimension.max_xyz': array([1190.125,  -82.775,  123.925]),
 'dendrite.node.dimension.bias_xyz': array([222.425, 201.6  ,  51.075]),
 'basal_dendrite.node.dimension.width': 222.4250000000011,
 'basal_dendrite.node.dimension.height': 201.60000000000127,
 'basal_dendrite.node.dimension.depth': 196.77500000000055,
 'basal_dendrite.node.dimension.min_xyz': array([ 967.7  , -284.375,  -72.85 ]),
 'basal_dendrite.node.dimension.max_xyz': array([1190.125,  -82.775,  123.925]),
 'basal_dendrite.node.dimension.bias_xyz': array([222.425, 201.6  ,  51.075]),
 'axon.tip.dimension.width': 1871.8500000000004,
 'axon.tip.dimension.height': 3010.35,
 'axon.tip.dimension.depth': 556.8250000000003,
 'axon.tip.dimension.min_xyz': array([  -98.8, -2929.8,  -370.6]),
 'axon.tip.dimension.max_xyz': array([1773.05 ,   80.55 ,  186.225]),
 'axon.tip.dimension.bias_xyz': array([1674.25 , 2849.25 ,  184.375]),
 'all_neurites.tip.dimension.width': 1871.8500000000004,
 'all_neurites.tip.dimension.height': 3010.35,
 'all_neurites.tip.dimension.depth': 556.8250000000003,
 'all_neurites.tip.dimension.min_xyz': array([  -98.8, -2929.8,  -370.6]),
 'all_neurites.tip.dimension.max_xyz': array([1773.05 ,   80.55 ,  186.225]),
 'all_neurites.tip.dimension.bias_xyz': array([1674.25 , 2849.25 ,  184.375]),
 'dendrite.tip.dimension.width': 222.4250000000011,
 'dendrite.tip.dimension.height': 201.60000000000127,
 'dendrite.tip.dimension.depth': 196.77500000000055,
 'dendrite.tip.dimension.min_xyz': array([ 967.7  , -284.375,  -72.85 ]),
 'dendrite.tip.dimension.max_xyz': array([1190.125,  -82.775,  123.925]),
 'dendrite.tip.dimension.bias_xyz': array([222.425, 201.6  ,  51.075]),
 'basal_dendrite.tip.dimension.width': 222.4250000000011,
 'basal_dendrite.tip.dimension.height': 201.60000000000127,
 'basal_dendrite.tip.dimension.depth': 196.77500000000055,
 'basal_dendrite.tip.dimension.min_xyz': array([ 967.7  , -284.375,  -72.85 ]),
 'basal_dendrite.tip.dimension.max_xyz': array([1190.125,  -82.775,  123.925]),
 'basal_dendrite.tip.dimension.bias_xyz': array([222.425, 201.6  ,  51.075]),
 'axon.compartment.dimension.width': 1961.0125000000007,
 'axon.compartment.dimension.height': 3100.5875,
 'axon.compartment.dimension.depth': 1416.1375,
 'axon.compartment.dimension.min_xyz': array([ -194.1  , -2918.725, -1205.275]),
 'axon.compartment.dimension.max_xyz': array([1766.9125,  181.8625,  210.8625]),
 'axon.compartment.dimension.bias_xyz': array([1572.8125, 2736.8625,  994.4125]),
 'all_neurites.compartment.dimension.width': 1961.0125000000007,
 'all_neurites.compartment.dimension.height': 3100.5875,
 'all_neurites.compartment.dimension.depth': 1416.1375,
 'all_neurites.compartment.dimension.min_xyz': array([ -194.1  , -2918.725, -1205.275]),
 'all_neurites.compartment.dimension.max_xyz': array([1766.9125,  181.8625,  210.8625]),
 'all_neurites.compartment.dimension.bias_xyz': array([1572.8125, 2736.8625,  994.4125]),
 'dendrite.compartment.dimension.width': 1961.0125000000007,
 'dendrite.compartment.dimension.height': 3100.5875,
 'dendrite.compartment.dimension.depth': 1416.1375,
 'dendrite.compartment.dimension.min_xyz': array([ -194.1  , -2918.725, -1205.275]),
 'dendrite.compartment.dimension.max_xyz': array([1766.9125,  181.8625,  210.8625]),
 'dendrite.compartment.dimension.bias_xyz': array([1572.8125, 2736.8625,  994.4125]),
 'basal_dendrite.compartment.dimension.width': 1961.0125000000007,
 'basal_dendrite.compartment.dimension.height': 3100.5875,
 'basal_dendrite.compartment.dimension.depth': 1416.1375,
 'basal_dendrite.compartment.dimension.min_xyz': array([ -194.1  , -2918.725, -1205.275]),
 'basal_dendrite.compartment.dimension.max_xyz': array([1766.9125,  181.8625,  210.8625]),
 'basal_dendrite.compartment.dimension.bias_xyz': array([1572.8125, 2736.8625,  994.4125]),
 'axon.bifurcation.dimension.height': nan,
 'axon.bifurcation.dimension.width': nan,
 'axon.bifurcation.dimension.depth': nan,
 'axon.bifurcation.dimension.min_xyz': array([nan, nan, nan]),
 'axon.bifurcation.dimension.max_xyz': array([nan, nan, nan]),
 'axon.bifurcation.dimension.bias_xyz': array([nan, nan, nan]),
 'all_neurites.bifurcation.dimension.height': nan,
 'all_neurites.bifurcation.dimension.width': nan,
 'all_neurites.bifurcation.dimension.depth': nan,
 'all_neurites.bifurcation.dimension.min_xyz': array([nan, nan, nan]),
 'all_neurites.bifurcation.dimension.max_xyz': array([nan, nan, nan]),
 'all_neurites.bifurcation.dimension.bias_xyz': array([nan, nan, nan]),
 'dendrite.bifurcation.dimension.height': nan,
 'dendrite.bifurcation.dimension.width': nan,
 'dendrite.bifurcation.dimension.depth': nan,
 'dendrite.bifurcation.dimension.min_xyz': array([nan, nan, nan]),
 'dendrite.bifurcation.dimension.max_xyz': array([nan, nan, nan]),
 'dendrite.bifurcation.dimension.bias_xyz': array([nan, nan, nan]),
 'basal_dendrite.bifurcation.dimension.height': nan,
 'basal_dendrite.bifurcation.dimension.width': nan,
 'basal_dendrite.bifurcation.dimension.depth': nan,
 'basal_dendrite.bifurcation.dimension.min_xyz': array([nan, nan, nan]),
 'basal_dendrite.bifurcation.dimension.max_xyz': array([nan, nan, nan]),
 'basal_dendrite.bifurcation.dimension.bias_xyz': array([nan, nan, nan]),
 'axon.num_nodes': 2456,
 'all_neurites.num_nodes': 3397,
 'dendrite.num_nodes': 940,
 'basal_dendrite.num_nodes': 940,
 'axon.num_branches': 179,
 'all_neurites.num_branches': 289,
 'dendrite.num_branches': 120,
 'basal_dendrite.num_branches': 120,
 'axon.num_tips': 169,
 'all_neurites.num_tips': 289,
 'dendrite.num_tips': 120,
 'basal_dendrite.num_tips': 120,
 'axon.mean_fragmentation': nan,
 'all_neurites.mean_fragmentation': nan,
 'dendrite.mean_fragmentation': nan,
 'basal_dendrite.mean_fragmentation': nan,
 'axon.max_branch_order': 0,
 'all_neurites.max_branch_order': 0,
 'dendrite.max_branch_order': 0,
 'basal_dendrite.max_branch_order': 0,
 'axon.num_outer_bifurcations': 0,
 'all_neurites.num_outer_bifurcations': 0,
 'dendrite.num_outer_bifurcations': 0,
 'basal_dendrite.num_outer_bifurcations': 0,
 'axon.total_length': 22977.13501045168,
 'all_neurites.total_length': 28858.00939732712,
 'dendrite.total_length': 5783.405108413997,
 'basal_dendrite.total_length': 5783.405108413997,
 'axon.total_surface_area': 169796.30610163006,
 'all_neurites.total_surface_area': 576437.5554815935,
 'dendrite.total_surface_area': 94071.25976294979,
 'basal_dendrite.total_surface_area': 94071.25976294979,
 'axon.total_volume': 137885.3630672354,
 'all_neurites.total_volume': 1418347.7263049642,
 'dendrite.total_volume': 167938.1662966555,
 'basal_dendrite.total_volume': 167938.1662966555,
 'axon.mean_diameter': 2.6006188925081433,
 'all_neurites.mean_diameter': 3.5045922873123345,
 'dendrite.mean_diameter': 5.657425531914893,
 'basal_dendrite.mean_diameter': 5.657425531914893,
 'axon.mean_parent_daughter_ratio': 1.07430268085247,
 'all_neurites.mean_parent_daughter_ratio': 1.1680862045868052,
 'dendrite.mean_parent_daughter_ratio': 1.1261357028487287,
 'basal_dendrite.mean_parent_daughter_ratio': 1.1261357028487287,
 'axon.max_euclidean_distance': 3361.3167542646147,
 'all_neurites.max_euclidean_distance': 3361.3167542646147,
 'dendrite.max_euclidean_distance': 1211.8040386650814,
 'basal_dendrite.max_euclidean_distance': 1211.8040386650814,
 'max_path_distance': 21.373640081768457,
 'early_branch_path': 0.5760635061828493,
 'mean_contraction': nan,
 'basal_dendrite.dendrite.overlap.above': 0.0,
 'basal_dendrite.dendrite.overlap.overlap': 1.0,
 'basal_dendrite.dendrite.overlap.below': 0.0,
 'dendrite.dendrite.overlap.above': 0.0,
 'dendrite.dendrite.overlap.overlap': 1.0,
 'dendrite.dendrite.overlap.below': 0.0,
 'axon.dendrite.overlap.above': 0.0,
 'axon.dendrite.overlap.overlap': 1.0,
 'axon.dendrite.overlap.below': 0.0,
 'basal_dendrite.axon.overlap.above': 0.1270358306188925,
 'basal_dendrite.axon.overlap.overlap': 0.04315960912052108,
 'basal_dendrite.axon.overlap.below': 0.8298045602605864,
 'dendrite.axon.overlap.above': 0.1270358306188925,
 'dendrite.axon.overlap.overlap': 0.04315960912052108,
 'dendrite.axon.overlap.below': 0.8298045602605864,
 'axon.axon.overlap.above': 0.0,
 'axon.axon.overlap.overlap': 1.0,
 'axon.axon.overlap.below': 0.0,
 'basal_dendrite.basal_dendrite.overlap.above': 0.0,
 'basal_dendrite.basal_dendrite.overlap.overlap': 1.0,
 'basal_dendrite.basal_dendrite.overlap.below': 0.0,
 'dendrite.basal_dendrite.overlap.above': 0.0,
 'dendrite.basal_dendrite.overlap.overlap': 1.0,
 'dendrite.basal_dendrite.overlap.below': 0.0,
 'axon.basal_dendrite.overlap.above': 0.0,
 'axon.basal_dendrite.overlap.overlap': 1.0,
 'axon.basal_dendrite.overlap.below': 0.0,
 'axon.node.moments.mean': array([8100.89725163, 1481.8711421 , 3041.58739821]),
 'axon.node.moments.std': array([ 548.29438592, 1015.93534595,  239.71152828]),
 'axon.node.moments.var': array([ 300626.73363552, 1032124.62714626,   57461.61679021]),
 'axon.node.moments.skew': array([-1.49463972,  1.47274862, -2.62686614]),
 'axon.node.moments.kurt': array([0.61521587, 0.32181039, 7.81303787]),
 'all_neurites.node.moments.mean': array([8072.73951281, 2016.52056962, 3092.57961437]),
 'all_neurites.node.moments.std': array([ 468.9743241 , 1221.78427798,  220.41264558]),
 'all_neurites.node.moments.var': array([ 219936.91666174, 1492756.82191868,   48581.73432959]),
 'all_neurites.node.moments.skew': array([-1.55061286,  0.30884535, -2.97477799]),
 'all_neurites.node.moments.kurt': array([ 1.48956331, -1.84925363, 10.52006684]),
 'dendrite.node.moments.mean': array([7999.24949468, 3411.95343085, 3225.66901596]),
 'dendrite.node.moments.std': array([44.23709914, 35.02212057, 30.29013021]),
 'dendrite.node.moments.var': array([1956.92094024, 1226.54892924,  917.49198825]),
 'dendrite.node.moments.skew': array([ 0.1546865 , -0.05942166, -0.21292968]),
 'dendrite.node.moments.kurt': array([-0.47036937,  0.05793513,  0.88034967]),
 'basal_dendrite.node.moments.mean': array([7999.24949468, 3411.95343085, 3225.66901596]),
 'basal_dendrite.node.moments.std': array([44.23709914, 35.02212057, 30.29013021]),
 'basal_dendrite.node.moments.var': array([1956.92094024, 1226.54892924,  917.49198825]),
 'basal_dendrite.node.moments.skew': array([ 0.1546865 , -0.05942166, -0.21292968]),
 'basal_dendrite.node.moments.kurt': array([-0.47036937,  0.05793513,  0.88034967]),
 'axon.tip.moments.mean': array([8182.92130178, 1288.64008876, 3083.24023669]),
 'axon.tip.moments.std': array([510.25922608, 906.76736357,  96.9590727 ]),
 'axon.tip.moments.var': array([260364.47779502, 822227.05163853,   9401.06177911]),
 'axon.tip.moments.skew': array([-1.9989749 ,  2.15153587, -0.23069053]),
 'axon.tip.moments.kurt': array([ 2.38061894,  2.72269615, -0.0741785 ]),
 'all_neurites.tip.moments.mean': array([8105.87474048, 2171.35700692, 3142.69679931]),
 'all_neurites.tip.moments.std': array([ 401.75893795, 1257.55755419,  105.11547926]),
 'all_neurites.tip.moments.var': array([ 161410.24422085, 1581451.002097  ,   11049.26398061]),
 'all_neurites.tip.moments.skew': array([-1.85329093,  0.0313061 , -0.72704499]),
 'all_neurites.tip.moments.kurt': array([ 3.52777148, -1.97721714, -0.14173403]),
 'dendrite.tip.moments.mean': array([7997.3675    , 3414.51666667, 3226.43145833]),
 'dendrite.tip.moments.std': array([52.56819077, 39.6691716 , 37.12362643]),
 'dendrite.tip.moments.var': array([2763.41468067, 1573.64317577, 1378.16363966]),
 'dendrite.tip.moments.skew': array([ 0.222898  , -0.16289955, -0.32449636]),
 'dendrite.tip.moments.kurt': array([-0.43350803,  0.26320655,  0.74040533]),
 'basal_dendrite.tip.moments.mean': array([7997.3675    , 3414.51666667, 3226.43145833]),
 'basal_dendrite.tip.moments.std': array([52.56819077, 39.6691716 , 37.12362643]),
 'basal_dendrite.tip.moments.var': array([2763.41468067, 1573.64317577, 1378.16363966]),
 'basal_dendrite.tip.moments.skew': array([ 0.222898  , -0.16289955, -0.32449636]),
 'basal_dendrite.tip.moments.kurt': array([-0.43350803,  0.26320655,  0.74040533]),
 'axon.compartment.moments.mean': array([8068.90239737, 2005.78467768, 3087.23589641]),
 'axon.compartment.moments.std': array([ 473.79868874, 1218.82535537,  227.5432284 ]),
 'axon.compartment.moments.var': array([ 224485.19744823, 1485535.2468946 ,   51775.92078956]),
 'axon.compartment.moments.skew': array([-1.52464721,  0.32815479, -2.88370718]),
 'axon.compartment.moments.kurt': array([ 1.36388044, -1.83335515,  9.68279285]),
 'all_neurites.compartment.moments.mean': array([8068.90239737, 2005.78467768, 3087.23589641]),
 'all_neurites.compartment.moments.std': array([ 473.79868874, 1218.82535537,  227.5432284 ]),
 'all_neurites.compartment.moments.var': array([ 224485.19744823, 1485535.2468946 ,   51775.92078956]),
 'all_neurites.compartment.moments.skew': array([-1.52464721,  0.32815479, -2.88370718]),
 'all_neurites.compartment.moments.kurt': array([ 1.36388044, -1.83335515,  9.68279285]),
 'dendrite.compartment.moments.mean': array([8068.90239737, 2005.78467768, 3087.23589641]),
 'dendrite.compartment.moments.std': array([ 473.79868874, 1218.82535537,  227.5432284 ]),
 'dendrite.compartment.moments.var': array([ 224485.19744823, 1485535.2468946 ,   51775.92078956]),
 'dendrite.compartment.moments.skew': array([-1.52464721,  0.32815479, -2.88370718]),
 'dendrite.compartment.moments.kurt': array([ 1.36388044, -1.83335515,  9.68279285]),
 'basal_dendrite.compartment.moments.mean': array([8068.90239737, 2005.78467768, 3087.23589641]),
 'basal_dendrite.compartment.moments.std': array([ 473.79868874, 1218.82535537,  227.5432284 ]),
 'basal_dendrite.compartment.moments.var': array([ 224485.19744823, 1485535.2468946 ,   51775.92078956]),
 'basal_dendrite.compartment.moments.skew': array([-1.52464721,  0.32815479, -2.88370718]),
 'basal_dendrite.compartment.moments.kurt': array([ 1.36388044, -1.83335515,  9.68279285]),
 'axon.bifurcation.moments.mean': array([nan, nan, nan]),
 'axon.bifurcation.moments.std': array([nan, nan, nan]),
 'axon.bifurcation.moments.var': array([nan, nan, nan]),
 'axon.bifurcation.moments.skew': array([nan, nan, nan]),
 'axon.bifurcation.moments.kurt': array([nan, nan, nan]),
 'all_neurites.bifurcation.moments.mean': array([nan, nan, nan]),
 'all_neurites.bifurcation.moments.std': array([nan, nan, nan]),
 'all_neurites.bifurcation.moments.var': array([nan, nan, nan]),
 'all_neurites.bifurcation.moments.skew': array([nan, nan, nan]),
 'all_neurites.bifurcation.moments.kurt': array([nan, nan, nan]),
 'dendrite.bifurcation.moments.mean': array([nan, nan, nan]),
 'dendrite.bifurcation.moments.std': array([nan, nan, nan]),
 'dendrite.bifurcation.moments.var': array([nan, nan, nan]),
 'dendrite.bifurcation.moments.skew': array([nan, nan, nan]),
 'dendrite.bifurcation.moments.kurt': array([nan, nan, nan]),
 'basal_dendrite.bifurcation.moments.mean': array([nan, nan, nan]),
 'basal_dendrite.bifurcation.moments.std': array([nan, nan, nan]),
 'basal_dendrite.bifurcation.moments.var': array([nan, nan, nan]),
 'basal_dendrite.bifurcation.moments.skew': array([nan, nan, nan]),
 'basal_dendrite.bifurcation.moments.kurt': array([nan, nan, nan]),
 'axon.calculate_number_of_stems': 11,
 'all_neurites.calculate_number_of_stems': 11,
 'dendrite.calculate_number_of_stems': 11,
 'basal_dendrite.calculate_number_of_stems': 11}

Save the features as h5 or csv file

We can also write these features to a file using the FeatureWriter. The FeatureWriter can write the features as a nested json or as a flat csv. If a feature returns a large amount of data, it can be output to a h5 file instead.

heavy_path = "test_features.h5"
table_path = "test_features.csv"

features_writer = fw.FeatureWriter(heavy_path, table_path)
features_writer.add_run("test", feature_extraction_run.serialize())
features_writer.write_table()

/Users/matthew.aitken/allen_dev/neuron_morphology/neuron_morphology/feature_extractor/feature_writer.py:54: H5pyDeprecationWarning: The default file mode will change to 'r' (read-only) in h5py 3.0. To suppress this warning, pass the mode you need to h5py.File(), or set the global default h5.get_config().default_file_mode, or set the environment variable H5PY_DEFAULT_READONLY=1. Available modes are: 'r', 'r+', 'w', 'w-'/'x', 'a'. See the docs for details.
  self.heavy_file = h5py.File(self.heavy_path, driver="core")
2020-02-25 13:13:49,879 root         WARNING  writing additional outputs to csv. See output json for record of selected features and marks