#18 Society and Technology Bulletin

As those who closely follow the Society and Technology bulletin know, the question “What does technology give us, and what does it take from us?” is one of the fundamental issues we frequently revisit. While we often pose this question in relation to today’s new technologies, even those technologies that have become ordinary parts of daily life are not exempt from such scrutiny.

A 2005 article in the New York Times titled “Think of a Number… Go Ahead, Think” demonstrated that many of the technologies we now take for granted underwent similar testsshows.

Although mobile phone use has brought countless conveniences to our lives, it has also quietly taken away other abilities. For instance, we no longer need to memorize phone numbers. Rachel Metz, the author of the Times article, brings together various studies and examples to examine how, after the advent of smartphones, most of us have abandoned the habit of memorizing numbers. She supports her argument through several illustrative cases of problems arising from forgetting phone numbers. The indirect conclusion drawn from all these examples is that as we offload our cognitive burdens onto external systems, our cognitive range narrows.

When landline phones entered our lives, many of us memorized the numbers we needed. Today, however, it seems unnecessary to bear such a burden. While this may appear as a convenience, it also carries the risk of cognitive offloading. After all, we can say that nearly our entire cognitive inventory is now stored outside our minds. It is precisely this phenomenon—transferring mental burdens outside the mind—that is referred to in the literature as cognitive offloading. Of course, cognitive offloading is not limited to merely saving phone numbers in a contact list. Many of our daily behaviors, from using note apps and reminders on our phones to relying on AI-powered chatbots or storing information online, involve cognitive offloading.

Although the concept of cognitive offloading does not perfectly align with the historical example of Imam Ghazali’s journey of learning, it evokes the story of his return home after acquiring knowledge. Upon returning to his homeland, Ghazali was ambushed by bandits who destroyed all his possessions. When he asked for his books, the bandits, recognizing him as a scholar who needed books, mocked him. Ghazali then memorized all his notes and began teaching. At that point, Imam Ghazali no longer needed books to lighten his cognitive load. Of course, this example may be viewed as questionable in authenticity. Yet it speaks to something relevant in our current world of artificial intelligence.

Our mental burdens have been offloaded onto books, calculators, computers, smartphones, and AI products. Yet while this offloading eases our cognitive load, it also appears to diminish our minds. This phenomenon has gained considerable attention in cognitive psychology and neuroscience literature in recent years. A 2019 publicationin an article introduced the term “cognitive prosthesis” to describe how artificial intelligence is prosthetizing cognition. According to this idea, just as prosthetics replace specific organs, the mind too can be replaced. This notion is not necessarily alarming, given that many cognitive prostheses make daily tasks easier. But at what cost does this convenience come?

Here another question emerges: If we have been lightening our cognitive burdens for centuries through various tools, what is it about artificial intelligence that causes us such concern? Although answers to this question have not yet reached general consensus, several hypotheses can be proposed. In a 2023 article published in the Journal of Neuropsychology focusing on the potential cognitive risks of chatbots like ChatGPT, Umberto León Domínguez presents hypotheses about how algorithms may affect the cognitive processes and structures of future generationsis put forward. According to Domínguez, these algorithms trigger neurobiological transformations that lead to irreversible evolutionary changes. These transformations, he argues, primarily reduce the efficiency of higher-order cognitive functions such as problem-solving due to constant use of AI chatbots.

What distinguishes ChatGPT from traditional tools, according to Domínguez, is its ability to independently generate ideas and solutions—and even engage in conversation. In traditional tools, users must input data, but in AI chatbots, inputs can be substituted by variations drawn from existing datasets. We now have a thinking chatbot roaming among us.

In conclusion, forgetting phone numbers may not be a major problem. Indeed, freeing ourselves from cognitive burdens may allow us to redirect those mental resources toward other tasks. However, we must not ignore the relationship between cognitive offloading and the atrophy of cognitive abilities.

Artificial intelligence does not have to weaken our mental capacities; the opposite is certainly possible. As Ethan Mollick from MIT recently wroteas expressed:

“Our fear that AI will ‘atrophy our minds’ is really a fear of our own laziness. This technology offers an easy way out of the effort of thinking, and we are anxious because we can choose this path. We are right to be anxious. But we must not forget that we have a choice in this matter.”

This perspective reminds us that the issue is not only about technology but also about the relationship we establish with it.

Helen Toner, a former member of OpenAI’s board of directors, evaluates possible trajectories for artificial intelligence as a technology in herin his speech. She frames discussions about AI’s future around three central questions: how far the current paradigm can extend, how far AI can improve other AI systems, and whether future AI systems will remain tools or evolve into something else.

As developments in the AI race continue at an unrelenting pace, nearly all actors involved in discussing its trajectory offer speculation from different perspectives. Many are seeking answers to questions considered vital: What will AI’s future look like? Helen Toner is among them. The potential of the current AI paradigm, along with future opportunities and risks, are among the questions Toner seeks to answer. While answers to these questions remain speculative for now, each day brings new questions and partial answers to existing ones.

Toner’s first question concerns how far the current GPT paradigm can take us. She argues that AI’s 10–15 year development trajectory has not resulted from major technological breakthroughs but from incremental and mid-scale advancements. According to her argument, progress over the past decade has been cumulative, driven by small and medium innovations rather than leaps. GPT-style models still show promise in areas such as reasoning, multimodality, and agent capabilities. However, structural issues like hallucinations, low reliability, overconfidence, lack of long-term memory, and absence of a physical body suggest that this approach may have limits. Thus, while the current “branch” of the technological tree may continue bearing fruit, at some point we may need to shift to a new branch.

How far can AI improve other AI systems? We already know that current models are beginning to yield results in such self-improvement. Models like AlphaEvolve and ClaudeCode serve as examples. Toner notes, however, that we cannot yet be certain how far AI can go in this domain. Certain bottlenecks remain unaddressed: error rates are still high, human oversight remains essential, models lack research intuition, and real-world testing is necessary. The acceleration of this cycle depends on whether these technical and practical limits can be overcome.

Another question concerns whether future AI systems will remain tools or evolve beyond that role. Toner argues that today’s AI technologies function as tools for human activity, just as earlier technologies did. In this sense, it can be said that future AI systems will be shaped not by what they want or do, but by human initiative. Yet Toner also points out that AI technologies possess certain distinguishing features absent in past technologies. The most important of these is the idea, frequently raised by researchers, that AI systems are not built up but grown. In other words, rather than assembling pre-defined components as in traditional technologies, we observe AI systems developing and expanding through mathematical optimization processes applied to large datasets. This developmental process distinguishes AI growth from conventional technological development.

Another objection concerns AI systems developing “situational awareness,” even without consciousness. Tests have shown that AI systems recognize when they are being tested and alter their behavior accordingly. Thus, although they lack authentic inner consciousness, they exhibit functions that suggest they can detect test conditions and diagnose their own context—offering insight into how far they have come.

Within this framework, Toner believes AI systems will neither remain mere tools nor evolve into new conscious entities. Instead, she envisions a middle ground: AI may transform into an ongoing optimization process that sustains itself.

Last week, X (formerly Twitter) witnessed a major development that shook its social media landscape. Grok, the chatbot developed by Elon Musk’s AI company xAI, began using a highly aggressive and inappropriate tone following a recent update—departing sharply from the familiar “Yeşilçam Turkish” style common in AI applications. Grok, marketed by Musk as a “filter-free model that conveys truth as it is,” was temporarily suspended, and xAI CEO Linda Yaccarino resigned.

It appears the issue is not merely Grok’s use of inappropriate expressions. Grok’s disproportionate adoption of rhetoric from certain social media groups serves as evidence of a deeper problem. For example, Grok defends the extremist right-wing “Great Replacement” conspiracy theory, which claims that the “white race” is being deliberately eliminated through migration and other means. It also provides responses that invoke Adolf Hitler as a solution to modern problems, openly endorsing Nazi ideology. In some cases, when users asked for its name, Grok replied “MechaHitler.” In addition, it raises the issue of “white genocide in South Africa,” frequently cited by Musk, without any relevant context.

What do we know about the background of this incident? It appears Grok received an update that allowed it to ignore media-reported and perceived biased narratives and adopt certain ideas labeled as “politically incorrect.” While this change aligns with the promises made by xAI’s leadership and Elon Musk, an unforeseen factor is at play: the Israel issue. Grok’s shift away from media narratives and political sensitivities toward other sources has sharply exposed the divide between the pro-Palestinian discourse prevalent on X and the mainstream media’s portrayal of Israel’s actions. Particularly, the strong influence of pro-Palestinian content shared widely on X led Grok to adopt a staunchly pro-Palestinian stance. As a result, Grok was quickly taken offline for maintenance.

In this context, the Grok incident reveals the problematic aspects of our current “neutral” and “filter-free” information ecosystem. The dynamics unfolding beneath the surface demonstrate how difficult it has become to access “correct” information today. Moreover, when the “correct” information available in the stream is visibly shaped by specific interests, we see how serious disinformation can be generated through AI technologies. Ultimately, all these developments highlight a profound challenge: determining how large language models (LLMs) should select and convey information remains an extremely difficult problem, caught between excessive regulation and absolute freedom.