Glossary
Deep Learning
What is Deep Learning?
Deep learning is a subfield of machine learning and artificial intelligence (AI) that focuses on the development of algorithms capable of learning complex patterns and representations from large datasets. These models, called deep neural networks, learn through multiple processing layers. Their artificial “neurons” draw loosely on biological ideas; they are not miniature brains. Deep learning has been widely applied to various domains, including natural language processing, computer vision, speech recognition, and neuroscience.
Key Concepts in Deep Learning
-
Neural Networks
Deep learning models use artificial neural networks: mathematical systems that transform inputs into outputs. These networks consist of layers of interconnected nodes or artificial neurons, with each neuron receiving input from other neurons and producing an output based on a specific activation function. Connection weights are adjustable mathematical parameters, rather than biological synapses. Training adjusts parameters to optimize a chosen objective. -
Deep Architecture
Deep learning models are characterized by their deep architecture, which consists of multiple layers of neurons stacked on top of each other. This hierarchical structure allows the model to learn increasingly complex and abstract representations of the input data as it passes through successive layers. Depth refers to multiple successive processing layers; there is no universal layer-count threshold separating every deep and shallow model. -
Training and Optimization
Deep learning models are trained using large datasets and optimization algorithms, such as stochastic gradient descent or adaptive learning rate techniques. During training, the model adjusts its weights to minimize a loss function, which measures the difference between the predicted output and the actual output. This process typically involves backpropagation, a method for efficiently computing the gradients of the loss function with respect to each weight by applying the chain rule.
Applications and Implications in Neuroscience
-
Neural Coding and Decoding
Researchers use deep learning to analyze neural data. A decoding model tries to infer a stimulus from recorded brain activity; an encoding model tries to predict recorded activity from a stimulus. A model’s success can help compare hypotheses about representation, without establishing that the brain performs the same calculations. -
Brain-Computer Interfaces
Deep learning models have been used in the development of brain-computer interfaces (BCIs), which enable direct communication between the brain and external devices. These systems try to translate recorded neural signals into commands for an external device. Their usefulness depends on the signals, task, and performance of the particular interface. -
Understanding the Brain
Comparing a network’s predictions with neural recordings gives researchers one way to test models of perception and cognition. Similar outputs do not by themselves show that a network and a brain work in the same way.