NeuroNLP is a query interface for fruit fly brain datasets. It enables in-depth explorations and investigations of the connectomic/synaptomic structure of fruit fly brain, using intuitive rule-based English queries. The NLP query interface enables searching and locating neurons and synaptic partners with a variety of attributes of query targets (refer to Search Bar section below). NeuroNLP can be accessed from any browser supporting WebGL. Try it on your smartphone/tablet!
The Medulla 7 Column dataset is currently available. For more information, see Shin-ya Takemura et al., Synaptic circuits and their variations within different columns in the visual system of Drosophila, Proceedings of the National Academy of Sciences Nov 2015, 112 (44) 13711-13716. The dataset contains: Neurons: 3D morphology, neuron name, type, data source, etc. All "segments" that are not fully traced in the fib25 dataset are also included. Their names start with "segment". However, they can be only queried using their body ID. Synapses: presynaptic and postsynaptic partners of a neuron, number of synapses and locations of synapses. Synapses of a neuron with fragments that have no morphology are bunched together.
NeuroNLP user interface (UI) consists of mainly 4 sections Header Bar: main menu including the Get Started guide, Create/Load Tags, access to the list of visualized neurons/synapses and a list neuropils. Visualization Panel: 3D visualization of queried neurons/synapses and the underlying neuropils. Information Panel: detailed information including synaptic partners of neuron displayed in the visualizer. Search bar: command line for natural language queries.
The Search Bar is the central querying interface. It supports natural language queries of neurons, synaptic partners, etc. By combining various attributes of query targets, you can create some very powerful queries. Click on the info button to the left of the search bar for more information. Also checkout the Demos!
The Header Bar provides access to the main menu that includes: Get Started: a menu for helping users get started with NeuroNLP. Tags: a menu for creating and loading tags. Tags are 3d snapshots of stored visualization panels. Neurons/Synapses: a menu displaying the visualized neurons and synapses, as well as a list of pinned neurons/synapses. Checkout the "Neurons/Synapses Menu" Demo (GUI Usage #1 in the Demos) Neuropils: a menu of neuropils. It is not used in the Medulla dataset. Cell Types: a list of cell types in the Medulla dataset. It provides an easy way to add different cell types directly, without using the search bar.
The visualization panel provides a interactive workspace of neuron skeletons in the context of relevant neuropils. It allows you to change view, select neurons, pin neurons, etc. To help navigating the visualized neuropils, neurons and synapses, the visualization panel provides a list of useful buttons for ease of interaction. For more details regarding the visualization panel, click on the button on the top right corner of the panel. Also checkout the GUI Usage section of the Demos.
Information panel is your major source of information regarding the individual neurons. You can view/add the pre- and post-synaptic partners of selected neuron. Check out the "Info Panel" Demo (GUI Usage #7 in the Demos).
The Search Bar is the central querying interface. It supports natural language queries of neurons, synaptic partners, etc. By combining various attributes of query targets, you can create some very powerful queries. Your queries should start with a verb; the verbs supported right now are: show: clear workspace and then show the queried neurons, add: add to the workspace the neurons queried, remove: remove from the workspace the queried neuron, keep: keep in the workspace only the neurons that meet the criterion of the query, hide: hide the neurons that meet the criterion of the query (this does not remove them from workspace, but reduce their visibility), pin: pin the neurons that meet the criterion of the query. Pinned neurons are automatically highlighted, and cannot be removed by the "trash can" button on top of the NeuroNLP window. unpin: unpin the neurons that meet the criterion of the query, color: color the neurons that meet the criterion of the query with a user defined color (can be hex color code, e.g., FF0000 for red), or these predefined colors. Additionally, the following single word commands are supported: clear: clear up the workspace, removing all neurons and synapses, undo: undo your last query. Next, we explain the rules for defining the criterion of the query, using the verb show as an example. show cell-type neurons: Shows the neurons of the cell type. Example: show Mi1 neurons, or simply show mi1. show $string$ neurons: Shows neurons with a name that contains the string. Example: show $T4a$ neurons, or simply show $T4a$, will query any neuron whose name contain the string *T4a*. show /rstring/r neurons: Show neurons whose name matches the regular expressing string (This requires some knowledge of how the neurons are named in each dataset). Example: show /rTm2-[A-C]/r neurons, or simply show /rTm2-[A-C]/r will show the PEG neurons that innervate PB glomerulus R1. show neurons with|that have inputs|outputs in neuropil/subregion: More specific then the previous query on the inputs or outputs. Examples: show neurons in column A with outputs in m10 show neurons presynaptic|postsynaptic to: Shows the neurons that are presynaptic or postsynaptic to the neurons defined after the word to. Exmples: show neurons presynaptic to $T4a-fb-home$, show Pm2 postsynaptic to Mi1 with at least 5 synapses, show presynaptic|postsynaptic neurons: Shows the neurons that are presynaptic or postsynaptic to the neurons already in workspace. Examples: show presynaptic neurons, show postsynaptic neurons with at least 10 synapses show postynaptic Dm1 show neurotransmitter neurons: Shows the neurons that express the neurotramsitter. Examples: show GABAergic neurons show cholinergic presynaptic neurons Other short-hands: show /:referenceId:[30465, 7892]: Shows the neurons whose referenceId in the original dataset is in the list. It can be used similar to \$ \\$ and regular expression and combined with other types of criteria. Examples: show /:referenceId:[30465, 7892] Coloring: if no criteria are specified, the color will be applied to the neurons added in the most recent query. For example, if you query: show A neurons, then add B neurons, color red will color B neurons red. color A neurons 0000FF will then color A neurons blue.
To use your mouse to navigate in the 3D visualization: Try out the "Pin/Select/Highlight" Demo (GUI Usage #3 in the Demos). Translation Right-click and then drag the cursor to translate the entire 3D scene. On mobile and tablets, the scene can be translated by dragging with three fingers. Rotation Left-click and then drag the cursor to rotate the entire 3D scene. On mobile and tablets, touch and drag with one finger to rotate the scene. Zoom Scroll up or down to zoom in or out the entire 3D scene. On mobile and tablets, pinch out or in to zoom out or in respectively. Highlight a Neuron/Synapse Hover over a neuron/synapse will highlight that neuron, with all other objects dimmed (except for those pinned). Brightness of highlighted objects and dimmed objects can be respectively adjusted in the Visualization Settings (see also Visualization Settings below). Select a Neuron/Synapse Hover over a neuron/synapse and Left-Click to retrieve the detail information in the Info Panel. On mobile and tablets, touch on the neuron with a single finger to select it. Pin a Neuron/Synapse Hover over a neuron/synapse and Double-Click to pin it in the scene. Pinned neurons will always be highlighted. Brightness of pinned objects and unpinned objects can be respectively adjusted in the Visualization Settings (see also Visualization Settings below). After pinning neurons of interest, you can use the remove unpinned neurons icon ( ) to restrict the workspace to neurons you are interested in. On mobile and tablets, select a neuron and then touch and hold on it for at least 750ms to pin it.
Try out the "Visualization Panel Buttons" Demo (GUI Usage #3 in the Demos). Here we list the function of each button: : Open visualization settings. : Open this menu. : Turn on/off stats performance monitor. : Take a screenshot of the current 3D visualization. The quality of the screenshot depends on the resolution of the display and the browser window size. : Reset view to original setting. : Zoom in onto the visualized (not hidden) neurons. : Show all neurons, synapses and neuropils. : Hide all unpinned neurons/synapses. : Remove all unpinned neurons/synapses.
Try out the "Graph View / Connectivity Buttons" Demo (GUI Usage #4 in the Demos). Three additional buttons in the Visualization Panel provide information about the graph of the visualized neural circuit: : Open the Graph View to generate a neuronal level graph of the neurons in the current workspace. The graph is visualized in 2D. Nodes representing each neuron has the same color as in the 3D visualization. Nodes can be reordered by dragging them. Hovering over a node will highlight the corresponding neuron in the 3D visualization. Clicking on the node hide/unhide morphology of the corresponding neuron. You can adjust the synapse threshold to remove connections that has less number of synapses than the value. : Open the Graph View to generate a cell type level graph of the neurons in the current workspace. Connections from all neurons of one cell type to all neurons of another cell type will be aggregated. Nodes can be reordered by dragging them. Hovering over a node will highlight the neurons of the corresponding cell type in the 3D visualization. Clicking on a node hide/unhide morphology of the neurons of the corresponding cell type. : Download the graph in CSV format, that includes the pre-synaptic neuron name, post-synaptic neuron name and the number of synapses on each row.
You can customize the visual effects in the Visualization Panel by using the Visualization Settings menu. Here is a break down of the options. Backgound Color: change the color of the background of the Visualization Panel. Neuron Rendering Rendering Mode: change the way neuron morphology is rendered. Default Neurite Radius: If the neuron skeleton does not contain radius information, this is the default value that will be used. Neurite Radius Range: The range of acceptable radius. Any radius outside of this range will be clipped to the nearest acceptable range. Default Soma Radius: If the soma of the neuron skeleton does not contain radius information, this is the default value that will be used. Soma Radius Range: The range of acceptable radius of soma. Any radius outside of this range will be clipped to the nearest acceptable range. Default Opacity: The opacity of visualized neurons/synapses when none of them are highlighted or pinned. Higlighted Neuron Opacity: The opacity of the neuron/synapse being highlighted. Low Opacity in Highlight Mode: When a neuron/synapse is highlighted, the opacity of the rest of the neurons/synapses. Pinned Neuron Opacity: The opacity of pinned neurons/synapses. Low Opacity in Pinned Mode: When some neurons/synapses are pinned, the opacity of the unpinned neurons/synapses. Ambient Light: Brightness of ambient light onto neurons/synapses. Directional Light: Brightness of directional light onto neurons/synapses. Neuropil Rendering Color: Color of the Neuropil meshes. Opacity: Opacity of the Neuropil meshes. Wireframe Opacity: Opacity of the wireframe on the Neuropil meshes. Screen-space ambient occlusion pass (SSAO): Whether to enable SSAO effect. Ambient Light: Brightness of ambient light onto neuropils. Directional Light: Brightness of directional light onto neuropils. Synapse Rendering Synapse Size: Synapses are renders as a sphere with a radius specific by the value of Synapse Size. Synapse Size Range: Range of valid synapse size if they are specified in the data. If the synapse size is outside of this range, it will be clipped. Synapse Opacity: The opacity of the synapses when no neuron/synapse is highlighted or pinned. PostProcessing Settings Fast approXimate Anti-Aliasing (FXAA): Better anti-aliasing. Bloom Pass: Create a glowing effect.