Natural Language Processing

Natural Language Processing (NLP) is a subfield of artificial intelligence (AI) that enables computers to analyze, process, and understand natural languages used by humans. This technology aims to make human-computer interaction more natural and efficient. The complexity of natural language, including semantic ambiguities and polysemy, such as these challenges, constitute important research topics in natural language processing.

The origins of natural language processing extend back to the early years of computer science and artificial intelligence. In 1950, Alan Turing's paper titled "Computing Machinery and Intelligence" introduced the Turing Test, a concept that questions whether machines can think like humans. This test aims to distinguish between a human and a machine based on their ability to engage in dialogue using natural language.

Between 1950 and 1960, NLP relied on rule-based systems. These systems attempted to interpret sentences by applying grammatical rules and syntactic analysis. In 1954, the Georgetown-IBM Experiment, conducted by IBM and Georgetown University, presented a prototype of an automated translation system using a very limited vocabulary. This demonstration generated significant enthusiasm about the potential of NLP at the time.

However, the late 1960s and 1970s ushered in a period known as the "AI Winter," during which academic and financial support declined due to the limitations of NLP systems and their failure to meet expectations. In the 1980s, new approaches emerged, placing greater emphasis on knowledge representation, semantics, and pragmatic analysis.

Starting in the 1990s, statistical approaches to natural language processing gained prominence. During this period, models such as Hidden Markov Models (HMM), Naive Bayes, and decision trees became widely used. These statistical methods began offering more flexible solutions in language processing by learning from data.

Beginning in the 2010s, deep learning-based approaches transformed the field of NLP. In particular, Google's Transformer architecture and models built upon it—such as BERT and GPT—sparked a major advancement in NLP applications. These models opened the way for advanced systems capable of understanding contextual meaning and performing syntactic and semantic analysis simultaneously.

Today, NLP is actively used across various domains in both academia and industry, playing a central role in human-computer interaction, information retrieval, and text-based decision making.

^{AI-generated representative image of "Natural Language Processing".}

For NLP systems to function successfully, they must perform analysis across multiple layers designed to translate the structural and semantic complexity of natural language into a form computers can process:

• Morphological Analysis: Words consist of a root and attached affixes. Morphological analysis determines the root of a word and identifies which affixes have been added. For example, the word "kitaplarımdan" is decomposed into "kitap" (root), "lar" (plural), "ım" (possessive), and "dan" (ablative case).
• Syntactic (Grammatical) Analysis: This analysis examines the structural relationships between words in a sentence, identifying elements such as subject, predicate, and object, and determining which words are connected to each other. This layer is typically represented using a parse tree.
• Semantic (Meaning-based) Analysis: This involves extracting meaning at the word or sentence level, going beyond literal definitions to interpret context. For example, the word "elma" can be understood as either a fruit or a technology company depending on context.
• Pragmatic and Contextual Analysis: This analysis determines the context in which speech or writing occurs—who said it, to whom, and with what intent. For instance, the intent behind the sentence "Bugün hava nasıl?" is identified as a request for information. This layer also addresses issues such as pronoun resolution (co-reference resolution), determining which entities pronouns refer to.

Each of these analyses works in tandem to help computers better understand language. In particular, deep learning approaches have enabled systems to learn many of these analyses simultaneously.

• Vocabulary: The set of terms used in a text or speech.
• Corpus: Collections of texts of similar type, such as film reviews or social media posts.
• Preprocessing: Steps applied to remove unwanted text, terms, and noise from a text. This is the first step in solving any NLP problem.
• Tokenization: The process of dividing a large text into smaller units called tokens. Each token is a meaningful unit of text.
• Embeddings: The process of converting each token into a vector representation before feeding it into a machine learning model. Embeddings can be created for words, word phrases, or characters.
• N-Grams: The representation of a text as sequences of n consecutive words or characters.
• Transformers: Deep learning architectures capable of parallel computation, designed to learn long-range dependencies in text.
• Parts of Speech (POS): The grammatical roles of words in a sentence (e.g., noun, verb).
• Parts of Speech Tagging: The process of labeling words with their grammatical roles, such as noun or verb.
• Stop Words: The process of identifying and removing words that contribute little semantic value, such as conjunctions.
• Normalization: The process of mapping similar terms to a canonical form.
• Lemmatization: The process of determining the base or dictionary form of a word.
• Stemming: Aims to reduce words to their root form, similar to lemmatization, but without using POS tags to guide the process.
• Feature Extraction: The process of identifying key words or phrases relevant to the task at hand, enabling classification of specific words or word groups.

The methods used in natural language processing have evolved and diversified over time. These methods can be broadly categorized into four main groups:

^{Generated by AI.}

Each method has its own advantages and limitations. Rule-based systems are transparent and controllable, while deep learning models are more comprehensive but have a "black box" structure. Depending on the application, these methods can be combined to create hybrid systems.

Natural language processing technologies have moved beyond being purely academic and have become integral parts of daily life, commerce, education, and public services. Below are the major application areas of NLP, described in detail:

1. Machine Translation (MT): One of the most well-known applications. Systems such as Google Translate, DeepL, and Microsoft Translator operate using statistical, rule-based, or neural network approaches. Current approaches use the Transformer architecture to produce context-aware and more accurate translations.
2. Voice Assistants and Speech Recognition: Systems like Siri, Alexa, and Google Assistant detect spoken sentences, convert them into text, interpret their meaning, and generate responses. These systems integrate both speech recognition and natural language understanding modules.
3. Text Mining and Information Extraction: Enables the extraction of meaningful and structured information from large text collections such as scientific papers, news archives, or social media. For example, automated information extraction systems can be used to survey medical literature on a specific disease.
4. Sentiment Analysis: Analyzes social media comments or customer feedback to classify them as positive, negative, or neutral. This plays a critical role for companies in managing brand perception.
5. Automatic Text Summarization: Used to automatically condense long texts such as news articles or reports into meaningful summaries. Two types of summarization are extractive and abstractive. It is particularly valuable for news agencies and academic journals.
6. Chatbots and Dialogue Systems: Chatbots encountered in e-commerce sites, banks, or public services understand user questions and provide automated responses. Advanced models can maintain context to generate more coherent dialogues.
7. Legal and Official Document Analysis: NLP techniques are applied to analyze court rulings, extract legal precedents, and classify legal documents.
8. Educational Technology: NLP-based systems enable automated feedback in language learning apps, vocabulary tracking, and reading comprehension tests.

These application areas continue to expand each year as technology and models improve. In the near future, more comprehensive language understanding systems are expected to enable even more effective applications in fields such as law, psychology, and creative writing.

Processing natural language with computers may appear simple at first glance, but the structure, meaning, and usage of language are complex. This complexity introduces several challenges in NLP applications:

• Semantic Ambiguity: The same word can have multiple meanings (e.g., "bank" can refer to a financial institution or the side of a river). This is a major issue that must be resolved in language processing.
• Syntactic Ambiguity: Ambiguities arising from sentence structure are common. In a sentence like "Kadın adamı gördü," the question of who did what to whom can be interpreted differently depending on context.
• Discourse Context: Sentences may be meaningful on their own but gain full meaning only as part of a larger text or dialogue. Pronoun resolution (e.g., "o," "onlar") is critical in this context.
• Lack of Universality in Language: Each language has unique structural features. In agglutinative languages like Turkish, morphological analysis is more complex. This requires systems to be redesigned for each language.
• Data Acquisition and Annotation: NLP systems require large volumes of diverse, annotated data to function accurately. Preparing such data is both time-consuming and costly.
• Domain-Specific Contexts: Some sentences can only be understood by someone with expertise in a particular field. For example, medical or legal texts require specialized knowledge.

Despite these challenges, advanced modeling techniques, pre-trained language models, and enriched data cultures are helping to mitigate these problems and develop more successful systems.

Behind the rapid development in NLP are open-source libraries and developer-friendly platforms. These tools are widely used in both academic research and industrial applications:

• NLTK (Natural Language Toolkit): A Python-based library ideal for students and academics, providing an excellent environment for fundamental NLP tasks such as tokenization, stemming, and tagging.
• spaCy: A high-performance, scalable Python library focused on efficiency. It excels in named entity recognition (NER), syntactic analysis, and context-aware applications and is recommended for industrial use.
• Stanford CoreNLP: Developed by Stanford University, this Java-based library stands out for its multilingual support and advanced semantic analysis capabilities.
• Hugging Face Transformers: A comprehensive library that simplifies the use of pre-trained models such as BERT, GPT, RoBERTa, and T5. It also provides infrastructure for fine-tuning and model deployment.
• OpenNLP and Flair: Alternative libraries notable for their multilingual support and efficient memory usage.

Thanks to these libraries, developers can build rapid applications using pre-trained models rather than training models from scratch. Particularly with transfer learning and fine-tuning techniques, more original applications can be developed.

A sequence of operations applied to a simple text sample:

• Cleaning using Regular Expressions (regex)
• Tokenization
• Removal of Stopwords (unimportant words)
• Lemmatization (reducing words to their base form)
• Creation of a Bag of Words
• Classification using a Machine Learning model

Turkish text sample:

Code:

Output:

Original Text: Bugün hava çok güzel, güneş parlıyor ve kuşlar ötüyor. Dün ise yağmur yağdı ve hava soğuktu.

1. Cleaned Text: bugün hava çok güzel güneş parlıyor ve kuşlar ötüyor dün ise yağmur yağdı ve hava soğuktu

2. Tokens: ['bugün', 'hava', 'çok', 'güzel', 'güneş', 'parlıyor', 've', 'kuşlar', 'ötüyor', 'dün', 'ise', 'yağmur', 'yağdı', 've', 'hava', 'soğuktu']

3. After Stopword Removal: ['bugün', 'hava', 'güzel', 'güneş', 'parlıyor', 'kuşlar', 'ötüyor', 'dün', 'yağmur', 'yağdı', 'hava', 'soğuktu']

4. Lemmatized: ['bugün', 'hava', 'güzel', 'güneş', 'parlıyor', 'kuşlar', 'ötüyor', 'dün', 'yağmur', 'yağdı', 'hava', 'soğuktu']

5. Bag of Words Matrix:

 [[1 1 1 1 1 1 1 1 1 1 1 1]]

Words: ['bugün' 'dün' 'güzel' 'güneş' 'hava' 'kuşlar' 'ötüyor' 'parlıyor' 'soğuktu' 'yağdı' 'yağmur']

6. Machine Learning Model Trained.

Sample Prediction: Test Text: 'hava güzel' -> Prediction: positive