DETR

Detection Transformer (DETR) is an innovative artificial intelligence model developed by Facebook AI in 2020 that adopts an end-to-end learning approach for object detection tasks. Unlike traditional object detection methods, DETR is the first model to center its architecture on the Transformer to predict the locations and classes of objects in an image.

^{DETR örnek şema (}Medium⁾

Traditional object detection systems typically involve a multi-stage processing pipeline. These stages include feature extraction, region proposal, classification, and bounding box refinement. Such systems commonly rely on CNN (Convolutional Neural Network) architectures and require specialized processing steps like Non-Maximum Suppression (NMS).

DETR simplifies this classic workflow by providing an end-to-end solution using only a CNN and a Transformer architecture. This eliminates the need for independent stages and handcrafted rules.

The DETR architecture consists of three key components:

CNN-based feature extraction: A convolutional neural network such as ResNet generates low-dimensional yet meaningful feature maps from the input image.

Transformer encoder-decoder structure: The Transformer module processes these feature maps. The encoder converts the input into meaningful vector representations, while the decoder uses “object queries” to evaluate these representations and perform object detection.

FFN (Feed-Forward Network): The decoder’s outputs are transformed into class labels and bounding box predictions for each object.

^{Örnek şema (}Medium⁾

At the core of DETR lies the Transformer architecture, inspired by the 2017 paper “Attention is All You Need” by Vaswani et al. The Transformer operates using self-attention, multi-head attention, and feed-forward layers. DETR is one of the first successful applications of this structure to image-based tasks.

DETR generates a fixed number of learned “object queries,” each representing a potential object in the image. The model performs all object predictions in parallel using these queries. Class and box assignments are determined using the Hungarian algorithm, implementing a matching strategy that eliminates redundant or duplicate predictions.

^{Macar algoritması (}Medium⁾

As a result, classical filtering operations such as Non-Maximum Suppression (NMS) are no longer required. The model directly assigns each prediction to an object or to the “no object” class.

DETR uses a loss function composed of two main components: class prediction and bounding box localization. The total loss is minimized by matching model predictions with ground truth objects. Special adjustments are made for the “no object” class to mitigate class imbalance during loss computation.

End-to-end learning: DETR simplifies the entire object detection pipeline by performing it within a single unified model.

Generalization capability: It can be easily adapted to different datasets without requiring complex handcrafted components.

Transformer advantages: It enables learning long-range dependencies and supports parallel processing.

Eliminates the need for NMS: Predictions are directly matched to objects, removing the need for post-processing filtering.

DETR can learn more slowly than classical approaches when detecting small objects. It requires long training times and large datasets. Additionally, latency issues may limit its use in real-time applications.

DETR is a pioneering work that transformed the paradigm of object detection by successfully applying the Transformer architecture. It overcomes the limitations of traditional methods by offering a simpler and more general solution. It is regarded as a foundational milestone in computer vision and artificial intelligence, paving the way for the next generation of detectors.