Edge AI (Edge Artificial Intelligence)

Edge AI (Uç Yapay Zeka) or Edge Intelligence (Uç Zeka - EI) refers to the practice of running artificial intelligence (AI) computations and algorithms on devices located at the network’s edge—close to where data is generated or users are present—rather than in centralized data centers.

This approach aims to integrate AI capabilities into mobile devices, Internet of Things (IoT) sensors, smart cameras, robots like real world devices. Fundamentally, it seeks to create distributed and autonomous intelligence capabilities closer to the data source by combining edge computing paradigms with AI techniques.

Although narrowly defined as running AI on edge devices, from a broader perspective it can also be viewed as a paradigm that fully leverages hierarchical collaboration between edge devices, edge servers, and the cloud to optimize AI model training and inference. The level of this collaboration may vary depending on how much data is transferred and how far it is sent.

^{Six Levels of Edge Intelligence (}Zhou et al.: Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing⁾

The origins of Edge AI can be traced back to the 1990s with the emergence of content delivery networks (CDNs) that hosted web and game videos on servers close to users. However, the development of Edge AI in its modern sense is closely tied to the evolution of computing models: the transition from centralized mainframe systems to personal computers, client-server (C/S) and browser-server (B/S) architectures, and subsequently to cloud computing.

Although the proliferation of cloud computing in the 2000s provided massive processing power and storage capacity in centralized data centers, challenges such as high latency, bandwidth bottlenecks, and data privacy concerns—especially due to the explosion of mobile devices and IoT—necessitated new approaches. In response, edge computing paradigms were developed to move computational power closer to the network’s edge:

• Cloudlet: Introduced in 2009 as a local, small-scale “data center in a box” concept for mobile devices.

• Fog Computing: Introduced in 2012 by Cisco as an intermediate layer between the cloud and devices, particularly for IoT applications.

• Mobile/Multi-access Edge Computing (MEC): Defined in 2014 by ETSI to provide low latency and high bandwidth in mobile networks, later expanded to include other access networks.

• Micro Data Centers (mDCs): A concept of hardware-oriented, portable, or ruggedized small data centers designed especially for industrial applications.

The emergence of Edge AI became possible through the development of these edge computing infrastructures, combined with increased efficiency in artificial intelligence algorithms—particularly deep learning—and the widespread adoption of IoT devices. It began to gain prominence in technology reports such as Gartner’s Hype Cycle around 2018 and has since attracted rapid interest in both academic and industrial domains. Today, Edge AI is recognized as a critical technology that delivers AI’s “last mile” to the network’s edge.

The rise of Edge AI stems from various technological and application-driven needs:

1. Low Latency: Bringing AI computations closer to where data is generated or to the user eliminates the delay of sending data to the cloud and waiting for results, enabling millisecond-level response times critical for time-sensitive applications such as autonomous vehicles, real-time video analysis, industrial control, and AR/VR.

2. Bandwidth and Cost Savings: Processing large volumes of raw data—especially video—at the edge instead of transmitting it to the cloud significantly reduces network traffic and associated communication costs. Only meaningful results or model updates may need to be sent to the cloud.

3. Privacy and Security: Processing sensitive data—such as health records, personal images, or industrial secrets—locally on devices or trusted edge servers minimizes the risk of data exposure during transmission and centralized storage, thereby enhancing privacy and data security. Techniques like Federated Learning provide additional safeguards.

4. Reliability and Accessibility: Edge devices can execute AI tasks locally even when network connectivity is intermittent or absent, ensuring operational continuity especially for critical infrastructure and industrial applications.

5. Unlocking Edge Data Potential: The massive volume of data generated by billions of IoT and mobile devices is predominantly produced at the network’s edge. Edge AI is an essential tool for extracting real-time insights and meaningful information from this data.

6. New Applications and Capabilities: Edge AI enables numerous new application scenarios previously impossible or impractical: advanced automation, smarter devices, personalized experiences, and context awareness are among them.

7. Scalability and Distribution: By distributing AI workloads from centralized cloud systems to distributed edge devices, overall system scalability can be improved and congestion at a single central point reduced.

For these reasons, the market size of Edge AI is expected to grow exponentially in the coming years.

Edge AI systems typically operate within a hierarchical structure comprising edge devices (data-collecting sensors, cameras, mobile phones, etc.), edge servers (more powerful computational units connected to access points, base stations, or local network gateways), and the cloud (for centralized storage, intensive model training, and global coordination). Within this architecture, AI model training and inference can be distributed in various ways:

• Cloud Training: The most common approach. Data is sent from the edge to the cloud, and models are trained centrally.

• Federated Learning: A privacy-focused approach where data remains on devices and only model updates—such as gradients or weights—are aggregated on a central server (cloud or edge server). This is especially important when working with sensitive data or when reducing communication costs is desired. Various variants exist (communication-efficient, resource-optimized, security-enhanced).

• Distributed Edge Training: Models are trained collaboratively across edge devices or edge servers, using either a central coordinator or peer-to-peer methods such as Gossip protocols.

• Transfer Learning / Knowledge Distillation: Knowledge learned from a large pre-trained model (teacher) is transferred to a smaller model (student) designed to run on edge devices, accelerating training and reducing resource requirements.

• On-Device Inference: The model runs entirely on the edge device. It offers the lowest latency and highest privacy but is constrained by the device’s resources. Model compression techniques play a critical role here.

• Edge Server Inference: The device sends data to a nearby edge server, inference is performed there, and the result is returned to the device.

• Device-Edge Co-inference: The model is split between the device and the edge server. The device runs the initial layers, sends compressed intermediate data to the edge server, and the remaining layers are executed there. Dynamically selecting the split point (e.g., Neurosurgeon, Edgent) can optimize performance.

• Edge-Cloud Co-inference: The device sends data to the edge, which may then forward it to the cloud if needed, or the model is split between edge and cloud. Used especially in scenarios requiring intensive computation.

• Early Exit: Exit points are added at intermediate layers of the model. For simple inputs, results are obtained from an early layer without running the full model, reducing latency.

• Edge Caching: Previously computed inference results or intermediate features for similar inputs are cached at the edge to avoid redundant computations.

Recently, a vision has emerged to make Edge AI systems autonomous using Large Language Models (LLMs, e.g., GPT). In this approach, the LLM acts as a controller in the cloud, understanding user requests in natural language, evaluating the capabilities of existing AI models, decomposing tasks into subtasks, selecting and coordinating appropriate models, and even automatically generating code for Federated Learning.

^{Federated Learning Process (Source:} Wang et al. 2019, In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning⁾

Edge AI has application potential across nearly every industry. Key application areas include:

• Smart Cities and Homes: Local intelligence in devices such as traffic management systems, smart parking, video analytics for public safety, energy management, voice assistants, smart thermostats, and security cameras.

• Autonomous Vehicles and Transportation: Real-time processing of sensor data for vehicle perception, immediate decision-making (e.g., emergency braking), route planning, and communication with other vehicles (V2V) or roadside units (RSU).

• Industrial IoT (IIoT) and Smart Manufacturing: Factory automation, predictive maintenance (monitoring machine conditions and predicting failures), quality control (defect detection via image processing on production lines), robotic control, and hazardous environment monitoring.

• Healthcare: Real-time patient monitoring via wearable devices, anomaly detection, early warning systems, and medical image analysis.

• Video Analytics and Surveillance: Performing tasks such as facial recognition, object detection, and crowd analysis on servers near cameras to reduce bandwidth requirements and latency.

• Retail and Marketing: Customer behavior analysis and delivery of personalized recommendations and advertisements.

• Augmented/Virtual Reality (AR/VR): Delivering low-latency, high-quality experiences that respond instantly to user movements and surroundings.

• Drones and Robots: Processing autonomous navigation, environmental perception, and task execution capabilities on-device or nearby.

The implementation of Edge AI is made possible by various technological advancements in hardware and software:

• Hardware: Specialized AI Accelerators: Chips designed to run AI models on edge devices efficiently with low power consumption. Examples: Google Edge TPU, Intel Movidius VPU/Nervana NNP, Huawei Ascend, Qualcomm Snapdragon (with NPU/APU), HiSilicon Kirin (NPU), MediaTek Helio (APU), NVIDIA Jetson (mobile GPU).

• Mobile CPUs and GPUs: Standard processors in smartphones and other edge devices can be used for lightweight AI tasks when paired with optimized AI libraries.

• FPGAs (Field-Programmable Gate Arrays): Programmable hardware offering low power consumption and flexibility, allowing customization for specific AI workloads.

• Edge Server Hardware: Servers integrated into micro data centers (mDCs) or base stations/network gateways provide higher computational power.

• Software and Algorithms: AI/ML Models and Algorithms: Deep learning models (CNNs, RNNs/LSTMs) and Deep Reinforcement Learning (DRL) form the foundation of Edge AI. Algorithms optimized for resource-constrained environments are essential.

• Model Compression Techniques: Methods that reduce model size and computational requirements to enable execution on edge devices: Weight Pruning, Data Quantization, Compact Architecture Design, Knowledge Distillation.

• Federated Learning: A framework for distributed, privacy-preserving model training.

• AI Frameworks and Libraries: AI development and execution environments optimized for edge devices: TensorFlow Lite, PyTorch Mobile, Core ML, ONNX Runtime, OpenVINO, CMSIS-NN, SNPE, etc.

• Edge Computing Platforms and Frameworks: Software used to manage, deploy, and orchestrate edge devices and applications: KubeEdge, Azure IoT Edge, AWS IoT Greengrass, EdgeX Foundry, Akraino, OpenNESS, etc.

• Network Technologies: 5G and beyond (6G), ultra-low latency (URLLC), Software-Defined Networking (SDN), Network Function Virtualization (NFV), and Network Slicing provide the necessary infrastructure for Edge AI.

Although Edge AI is a rapidly evolving field, significant challenges remain to be addressed, along with promising promises for future research directions:

• Resource Constraints: Limited computational power, memory, storage, and battery life of edge devices make running complex AI models challenging. Energy efficiency is a critical concern.

• Algorithm and Model Management: Designing, compressing, deploying, updating, and managing AI models tailored to diverse hardware and application requirements is complex. Compatibility issues may arise across heterogeneous devices.

• Security and Privacy: The distributed nature of edge devices makes them more vulnerable to security threats compared to centralized systems—such as data poisoning, model theft, and unauthorized access. Ensuring data privacy (compliance with regulations like GDPR) and developing trustworthy mechanisms (e.g., blockchain) is essential.

• Network Management and Optimization: Dynamic network conditions (latency, bandwidth fluctuations), device mobility, and coordination of large numbers of devices require intelligent management mechanisms for resource allocation (computation, communication, caching) and task scheduling. The concept of “Intelligent Edge”—using AI to optimize the edge network itself—is a major direction in this area.

• Standardization and Interoperability: Common standards and open platforms are needed to ensure seamless interoperability among hardware and software components from different vendors.

• Data Management: Efficiently collecting, labeling (especially for supervised learning), filtering, and processing the vast amounts of data generated at the edge is challenging. Data quality and non-IID (non-independent and identically distributed) data distributions pose difficulties for methods like Federated Learning.

• Autonomous Systems: There is a growing trend toward autonomous Edge AI systems that can self-organize, adapt, and optimize, supported by technologies like LLMs.

• Incentive Mechanisms: In a distributed ecosystem, economic models must be developed to incentivize different stakeholders—such as device owners and edge server providers—to share resources and collaborate.