1. EXPLORE - Brain Structure & Topology Analysis

The Explore tab allows you to analyze real brain MRI data, connectivity between brain regions, and the topological characteristics of each region.

Left Panel: MRI Viewer

• Brain Slice Viewing: Explore the brain through three different planes (Axial, Coronal, Sagittal).
• Mouse Wheel: Scroll through slices to view different parts of the brain.
• Atlas Overlay: Toggle the colored brain region overlay on and off.
• Overlay Alpha: Adjust the transparency of the overlay to better view both the raw MRI and atlas together.
• Region Click: Click on a specific region in the MRI image to display information about that brain region in the panel, and the corresponding node will be highlighted in the 3D network on the right.

Right Panel: 3D Brain Network & Topological UMAP

• 3D Network Visualization: Brain regions are represented as nodes in 3D space, with connections between regions displayed as lines.
• Node Click: Clicking on a node displays detailed information about that region and its UMAP point cloud.
• UMAP Point Cloud: Visualizes neural activity patterns of each brain region in 3D topological space.
  • CA1: Helix topology - temporal dynamics
  • LGd: Loop topology - periodic patterns
  • RSP: Manifold topology - complex state space
  • SC: Cluster topology - discrete state transitions
  • VISp: Trajectory topology - smooth neural state path
• Mouse Drag: Rotate the 3D network to observe from various angles.
• Mouse Wheel: Zoom in/out to examine detailed connectivity.
• Reset View: Return to the initial viewpoint.

Key Features

• MRI-Network Integration: Regions clicked in the MRI are automatically highlighted in the 3D network.
• Region Information Panel: View anatomical location, structural information, and connected regions for each brain area.
• Real-time Visualization: Based on actual MRI data and structural connectivity data from the Allen Brain Atlas.

2. GENERATE - Visual Perception Simulation

The Generate tab simulates and displays the visual scene that a person would perceive based on a configured brain state, rendered as a point cloud.

Left Panel: RealSense LiDAR Viewer

• GENERATE Button: Generates a point cloud of the visual scene based on the currently configured brain state.
• Connect RealSense: Connect an Intel RealSense camera to receive real-time depth data.
• Color Mode: Switch between depth visualization modes (depth coloring/actual color).
• Depth Range: Set the depth range to capture (adjust minimum/maximum distance).
• Point Cloud Visualization: Depth data is rendered as points in 3D space in real-time.

Right Panel: Brain State Network

• Brain Region Nodes: Each node represents a specific brain region and can be clicked to activate/deactivate.
• Network Connections: Show or hide connections between brain regions.
• Show Labels: Display or hide node labels (region names).
• Node Details Panel:
  • Selected node's ID and point count
  • Point Size: Adjust the size of points in the point cloud
  • Opacity: Adjust the transparency of the point cloud

How It Works

• Brain State Configuration: The combination of activated brain regions in the right network defines the current brain state.
• Visual Generation: Based on the configured brain state, the neural network renders the potential visual scene as a point cloud.
• Real-time Updates: When a RealSense camera is connected, the point cloud updates in real-time with actual environmental depth data.
• Procedural Generation: Even without a camera, various geometric shapes (sphere, torus, helix, wave, etc.) can be generated as point clouds.

General Tips

• Exploration: Use both tabs together to understand both the structural and functional outputs of the brain.
• Performance: Large point clouds may affect frame rate depending on your GPU performance.
• Data: Visualizations are based on actual neural activity data from the Allen Brain Observatory.