Suggested study time: 38 minutes • Beginner level • Original rewrite based on Microsoft Learn objectives
By João Ricardo Dutra••Complete original content
1. Introduction to computer vision
Computer vision is a core AI field that enables applications to process visual information. Autonomous vehicles detect traffic and pedestrians, smart checkouts identify products, and doorbell cameras detect visitors.
Computers do not have biological eyes, but they can process pixels from photographs, videos, and live feeds. Software uses those values to emulate parts of human perception, extract meaning, and take action.
The original module offers video or text-and-image learning. The written version includes additional detail and can supplement the videos.
2. Image classification
Image classification is one of the oldest vision techniques. A model trained on many labeled examples analyzes visual features in a new image and predicts a text label for its main subject.
At a grocery checkout, produce on a scale can be photographed and classified as an apple, orange, banana, or another item. Training requires many images with correct names; the application can then combine the predicted class with weight to calculate the price.
tasks progress from one label to location, pixel-level classes, and semantic interpretation.
3. Object detection and semantic segmentation
When a checkout must find several items at once, object detection examines multiple regions. Its result includes every predicted class and the coordinates of a rectangular bounding box around each object.
Semantic segmentation provides a more exact location by assigning every pixel to an object category. The resulting mask follows object shapes instead of merely surrounding them.
How three fundamental tasks differ.
Task
Output
Question
Classification
One label, usually with a probability.
What is the main subject?
Object detection
Classes and bounding-box coordinates.
Which objects are present and where?
Semantic segmentation
A class for every pixel.
Which exact region belongs to each category?
4. Contextual image analysis
Recent multimodal models learn relationships between visible objects and descriptive text. They can semantically interpret a scene, recognize objects and actions, produce captions, and suggest relevant tags.
A photo can therefore receive more than the label “apple”; the model may generate “a person eating an apple” by connecting the person, food, and action.
5. Images as pixel arrays
To a computer, an image is a numeric array. In a 7 × 7 grayscale image, each position is a pixel: 0 is black, 255 is white, and intermediate values are shades of gray. Seven rows and seven columns define its resolution.
This is a two-dimensional x-y array with one channel. Color images normally have three matrices, or channels, for red, green, and blue (RGB). Each final pixel combines three intensities.
In the source example, purple uses red 150, green 0, and blue 255. The yellow center uses red 255, green 255, and blue 0.
Resolution defines rows and columns; RGB channels combine color intensities.
6. Filters and convolution
Filters modify pixels to create effects or reveal features. They are defined by small weight arrays called kernels. A 3 × 3 kernel covers a patch; corresponding pixel and weight values are multiplied and summed to create one value in a new array.
Laplace kernel
-1 -1 -1
-1 8 -1
-1 -1 -1
In the first patch, the weighted sum is -255 because the final position contains 255. After moving one pixel right, two 255 values contribute and the sum becomes -510. Each result is appended to the output array.
Convolution continues across the image. Values outside 0–255 are adjusted; the border lacks a complete kernel, so padding, usually zero, is added. The new array represents a transformed image.
The filter emphasizes sharp transitions between neighboring pixels.
This Laplace filter highlights edges. Other kernels blur, sharpen, invert color, and create other effects. Moving a kernel across an image gives the operation its name: convolutional filtering.
7. Convolutional neural networks
Image editors use filters for appearance, while computer vision often needs actionable meaning. A convolutional neural network (CNN) learns filters that produce numeric feature maps and feeds those features to a deep network for label prediction.
For fruit classification, kernel weights begin randomly. Training compares predictions with known labels and adjusts both filter and network weights until useful visual features separate apples, bananas, and oranges.
Supply labeled images, such as 0 for apple, 1 for banana, and 2 for orange.
Use convolutional layers to generate feature maps; pooling can downsize maps and emphasize important patterns.
Flatten the maps into one numeric vector.
Feed the vector to a fully connected neural network.
Apply softmax or a similar function to output one probability per class, such as [0.2, 0.5, 0.3].
Learned filters extract features, and a connected network turns them into probabilities.
For a banana, the desired target is [0.0, 1.0, 0.0]. Prediction error produces the loss; the connected weights and filter kernels change to reduce it. Multiple epochs learn a useful parameter set, which is saved for inference on unlabeled images.
Real CNNs usually contain several convolution, pooling, constraint, and transformation layers. The simplified view emphasizes the principle: filters create numeric features, and a neural network predicts a label.
8. Transformers and semantic language models
CNNs powered vision for many years and remain the basis of complex tasks such as object detection, which combines CNN features with regions of interest. In NLP, transformers introduced another successful neural architecture.
Transformers process huge text collections, encode tokens as vector embeddings, and use attention to reflect how each token relates to its context. Dimensions capture linguistic attributes; terms used in similar contexts tend to have aligned vectors.
This semantic vocabulary supports analysis, translation, and generation. Production encoders use many dimensions and complex linear algebra, but the key idea is that encoding preserves relationships among entities.
9. Transformers and patches
A Transformer (ViT) applies the transformer pattern to visual data. Instead of text tokens, it extracts image patches, linearizes their pixel values, and creates one embedding per patch.
Attention measures contextual relationships among patches. Embeddings capture color, shape, contrast, texture, and other visual properties, creating a multidimensional feature map learned from training images.
Features seen in similar settings receive related directions. Hats and heads frequently occur together, for example. The model has no human concept of either object but can infer a semantic relationship between their visual characteristics.
A ViT builds contextual relationships between image regions.
10. Multimodal models
A language transformer creates a linguistic vocabulary, and a vision transformer creates a visual one. Training on paired images and descriptions allows their encoders to be combined in a multimodal model.
Cross-modal attention aligns image and text embeddings in a shared vector space. For an unseen image, the model recognizes visual patterns, searches for connected language, and can produce a description such as “a person in a park with a hat and a backpack”.
A shared space links text and visual features for captions, tags, and answers.
11. Image and video generation
The multimodal architecture that answers questions about images can also synthesize an image from a natural-language prompt. Learned links between words and visual features guide the composition.
Many modern generators use diffusion. They begin with random pixels and iteratively remove noise. After each step, the partial image is compared with the prompt until a coherent scene appears. “A dog carrying a stick in its mouth” progresses from noise to vague forms and finally the requested image.
Iterative denoising moves the image toward the requested semantic features.
Video generation can use the same principle but must also model physical behavior and time: a walking dog should keep its feet on the ground, and frames must form a logical sequence.
12. Exercise and assessment
The exercise uses a vision model in Chat Playground to identify and describe image contents.
The real interface capture is preserved as PNG; conceptual diagrams were recreated as SVG.
