My Blog.

Describe the self-organization map (SOM) algorithm and explain how it can be used for feature mapping.

Self-Organizing Map (SOM) Algorithm

Self-Organizing Maps (SOMs), also known as Kohonen maps, are a type of artificial neural network used for Unsupervised LearningUnsupervised LearningUnsupervised Learning Definition Unsupervised Learning is a type of machine learning where the algorithm is trained on unlabeled data. The goal is to infer the natural structure present within a set of data points. Unlike supervised learning, there are no predefined labels or outcomes, and the system tries to learn the patterns and the structure from the data. Key Concepts Unlabeled Data:** Data that does not have associated labels or target values. Clustering:** Grouping a set of objects in. They are particularly useful for dimensionality reduction, clustering, and feature mapping. The main idea behind SOMs is to produce a low-dimensional (typically 2D), discretised representation of the input space of the training samples.

Structure of a Self-Organising Map

  1. Grid of Neurons: SOMs consist of neurons arranged in a grid. Each neuron has a weight vector of the same dimension as the input vectors.

  2. Topology: The neurons are typically arranged in a rectangular or hexagonal grid, and the arrangement defines the topology of the map.

SOM Algorithm Steps

  1. Initialization:

    • Initialize the weight vectors of the neurons randomly or by sampling from the input data distribution.

  2. Input Presentation:

    • Present an input vector to the SOM.

  3. Best Matching Unit (BMU) Identification:

    • Find the neuron whose weight vector is closest to the input vector. This neuron is called the Best Matching Unit (BMU). [$$ \text{BMU} = \arg\min_j | \mathbf{x} - \mathbf{w}_j | $$] where ( \mathbf{x} ) is the input vector and ( \mathbf{w}_j ) is the weight vector of neuron ( j ).

  4. Weight Update:

    • Update the weight vectors of the BMU and its neighboring neurons to move closer to the input vector. The update rule is: [$$ \mathbf{w}_j(t+1) = \mathbf{w}j(t) + \eta(t) \cdot h{j,\text{BMU}}(t) \cdot (\mathbf{x}(t) - \mathbf{w}j(t)) $$] where ( \eta(t) ) is the learning rate, ( h{j,\text{BMU}}(t) ) is the neighborhood function that decreases with the distance from the BMU, and ( t ) is the time step.

  5. Neighbourhood Function:

    • The neighbourhood function ( h_{j,\text{BMU}}(t) ) defines the influence of the BMU on its neighbours. A common choice is the Gaussian function: [$$ h_{j,\text{BMU}}(t) = \exp\left(-\frac{| r_j - r_{\text{BMU}} |^2}{2\sigma^2(t)}\right) $$] where ( r_j ) and ( r_{\text{BMU}} ) are the positions of neuron ( j ) and the BMU on the grid, respectively, and ( \sigma(t) ) is the width of the neighborhood, which decreases over time.

  6. Repeat:

    • Repeat the process for a number of iterations or until the map converges.

Feature Mapping with SOM

Feature Mapping with SOM involves projecting high-dimensional data into a low-dimensional space while preserving the topological relationships of the input data. This process can be visualized as follows:

  1. Training the SOM:

    • During training, the SOM learns to organize the input data by adjusting the weight vectors of the neurons based on the input patterns. The weight vectors become prototypes that represent clusters of similar input data.

  2. Mapping Input Data:

    • After training, each input vector can be mapped to the neuron (BMU) with the closest weight vector. This mapping creates a 2D representation of the high-dimensional data.

  3. Visualisation:

    • The 2D grid of neurons can be visualized to show how input data is distributed across the map. Clusters of similar data points will be mapped to nearby neurons, revealing the structure of the data.

Example of SOM in Feature Mapping

Consider a dataset of handwritten digits, where each digit image is represented by a high-dimensional vector of pixel values. By training a SOM on this dataset:

  1. Initialisation: The SOM is initialized with a grid of neurons, each with a random weight vector.

  2. Training: The SOM is trained using the images of digits. Each image is presented to the SOM, and the weight vectors of the BMU and its neighbors are updated.

  3. Feature Mapping: After training, each neuron in the SOM represents a prototype digit. Similar digits (e.g., all variations of the digit "5") will map to nearby neurons on the grid.

  4. Visualisation: The 2D grid can be visualized, showing how different digits are organized. Neurons representing similar digits will be close to each other, revealing the underlying structure of the digit dataset.

Applications

  • Data Visualisation: SOMs can be used to visualise high-dimensional data in a 2D or 3D space, making it easier to understand and interpret complex datasets.
  • Clustering: By organising similar data points together, SOMs can be used for clustering tasks in various fields such as bioinformatics, marketing, and finance.
  • Feature Extraction: SOMs can be used to extract features from raw data, reducing dimensionality and highlighting important patterns.

Conclusion

Self-Organizing Maps are a powerful tool for feature mapping and dimensionality reduction. By preserving the topological relationships of the input data, SOMs provide a meaningful low-dimensional representation that can be used for visualization, clustering, and feature extraction. The algorithm's ability to organize and cluster data makes it particularly useful for exploring and understanding complex datasets.