Katary - Software Factory - Artificial Intelligence and Intellectual Debt

Artificial Intelligence and Intellectual Debt

Christian Cabrera Jojoa

Assistant Research Professor

Department of Computer Science and Technology

University of Cambridge

chc79@cam.ac.uk

Katary - Software Factory - Artificial Intelligence and Intellectual Debt

The AI Narrative

Katary - Software Factory - Artificial Intelligence and Intellectual Debt

AI History - Machine Learning Age (2001 - present)

1940 1950 1960 1970 1980 1990 2000 2010 2020 2030 First AI Winter (1974-1980) Second AI Winter (1987-1994) Big Data (2000-2012) Artificial Neuron (McCulloch & Pitts, 1943) Information Theory (Shannon, 1948) Cybernetics (Wiener, 1948) Updating Rule (Hebbian, 1949) Computing Machinery and Intelligence (Turing, 1950) SNARC (Minsky, 1951) AI Term (Dartmouth Workshop, 1956) GPS (Newell & Simon, 1957) Advice Taker (McCarthy, 1958) Back-Propagation (Kelley, 1960) Perceptrons (Rosenblatt, 1962) ELIZA (MIT, 1966) ALPAC Report (USA, 1966) The DENDRAL (Buchanan, 1969) Perceptrons Book (Minsky & Papert, 1969) PROLOG (1972) MYCIN (Stanford, 1972) Lighthill Report (UK, 1973) FRAMES (1975) Hopfield net (1982) R1 (McDermott, 1982) Parallel Distributed Processing (Rumelhart & McClelland, 1986) Bayesian Networks (Pearls, 1988) Reinforcement Learning (Sutton, 1988) Image Recognition (LeCun et al., 1990) Deep Blue beats Kasparov (IBM, 1997) Deep Learning (Hinton, 2006) Watson wins Jeopardy (2011) AlexNet (Krizhevsky, 2012) GANs (Goodfellow, 2014) AlphaGo beats Lee Sedol (DeepMind, 2016) Transformer (Vaswani, 2017) AlphaFold (DeepMind, 2018) GPT-1 (OpenAI, 2020) BERT (Google, 2019) Chinchilla (DeepMind, 2022) ChatGPT (OpenAI, 2022) LLaMA (Meta AI, 2023) Claude 2 (Anthropic, 2023) phi-3 (Microsoft, 2024) Gemini 1.5 (Google DeepMind, 2024) Qwen3 (Alibaba, 2025) R1 (DeepSeek) 2025)
Katary - Software Factory - Artificial Intelligence and Intellectual Debt

The AI Narrative

The Economist
The economics of superintelligence - The Economist (26th July - 8th August 2025)

"AI will be capable of generating novel insights next year." (Altman, 2025)


"Self-improving AI will create a super intelligence." (Musk, 2025)


"When I look at the data, I see many trend lines up to 2027." (Clark, 2025)


"There is a 10-20% chance that the technology will end in human extinction." (Hinton, 2025)


"Our relationship with future AI systems is that we are going to be their boss." (LeCun, 2025)

Katary - Software Factory - Artificial Intelligence and Intellectual Debt

The AI Narrative

"Machines will be capable, within twenty years, of doing any work a man can do." (Simon, 1965)


"In from three to eight years we will have a machine with the general intelligence of an average human being." (Minsky, 1970)


"In medicine, management, and the military — indeed in most of the world's work — the daily tasks are those requiring symbolic reasoning with detailed professional knowledge." (Feigenbaum, 1982)

"AI will be capable of generating novel insights next year." (Altman, 2025)


"Self-improving AI will create a super intelligence." (Musk, 2025)


"When I look at the data, I see many trend lines up to 2027." (Clark, 2025)


"There is a 10-20% chance that the technology will end in human extinction." (Hinton, 2025)


"Our relationship with future AI systems is that we are going to be their boss." (LeCun, 2025)

Katary - Software Factory - Artificial Intelligence and Intellectual Debt

The AI Narrative

You'll own slaves by 1965 (1957)
You'll own "slaves" by 1965 (1957) - https://medium.com/@theo/do-we-need-robot-rights-in-the-age-of-artificial-intelligence-690b9951bae0

"AI will be capable of generating novel insights next year." (Altman, 2025)


"Self-improving AI will create a super intelligence." (Musk, 2025)


"When I look at the data, I see many trend lines up to 2027." (Clark, 2025)


"There is a 10-20% chance that the technology will end in human extinction." (Hinton, 2025)


"Our relationship with future AI systems is that we are going to be their boss." (LeCun, 2025)

Katary - Software Factory - Artificial Intelligence and Intellectual Debt

The AI Narrative

AI just could be a smart buy (1987)
AI just could be a smart buy (1987)
The Economist
The economics of superintelligence - The Economist (26th July - 8th August 2025)
Katary - Software Factory - Artificial Intelligence and Intellectual Debt

The Scientific Method

Critical Thinking
Critical Thinking (Designed by freepik.com)

"It seems to me what is called for is an exquisite balance between two conflicting needs: the most skeptical scrutiny of all hypotheses that are served up to us and at the same time a great openness to new ideas. Obviously those two modes of thought are in some tension. But if you are able to exercise only one of these modes, whichever one it is, you're in deep trouble. (The Burden of Skepticism, Sagan, 1987)

Katary - Software Factory - Artificial Intelligence and Intellectual Debt

The "Technocentric" View

Single Model
ML System?
https://xkcd.com/1838/, CC BY-NC 2.5 , via XKCD
Katary - Software Factory - Artificial Intelligence and Intellectual Debt

The Systems View

ML-based system in context
Katary - Software Factory - Artificial Intelligence and Intellectual Debt

The Systems View

AI-based software systems are data-driven. Unlike in traditional systems, developers cannot fully predefine their behaviour. ML components learn such behaviour from data, operating as black boxes that propagate uncertainty into complex software.

AI System
Katary - Software Factory - Artificial Intelligence and Intellectual Debt

The Systems View

AI-based software systems are data-driven. Unlike in traditional systems, developers cannot fully predefine their behaviour. ML components learn such behaviour from data, operating as black boxes that propagate uncertainty into complex software.

AI System
Katary - Software Factory - Artificial Intelligence and Intellectual Debt

Intellectual debt

Intellectual Debt: Practitioners deploy data-driven systems that work in practice, but do not fully understand their inner workings. This threatens transparency, safety, and trust, increasing risks of AI's negative social impact (Zittrain, 2022).

Katary - Software Factory - Artificial Intelligence and Intellectual Debt

The Systems Engineering Approach

Katary - Software Factory - Artificial Intelligence and Intellectual Debt

The Systems Engineering Approach

The systems engineering approach is better equipped than the ML community to facilitate the adoption of this technology by prioritising the problems and their context before any other aspects.

Katary - Software Factory - Artificial Intelligence and Intellectual Debt

The Systems Engineering Approach

Systems Thinking
Process Model
Systems views: Defining the problem from different perspectives
Top-down analysis: Divide and conquer approach
Agility systems: Flexible architectures and solutions
Variant creation: Assessing solution alternatives
Systems dynamics: Models that show systems evolution
Problem solving cycle: Following a methodology
Katary - Software Factory - Artificial Intelligence and Intellectual Debt

The Systems Engineering Approach

AI Adoption

A survey of research works that apply systems engineering principles to address these challenges when deploying AI-based systems.

Katary - Software Factory - Artificial Intelligence and Intellectual Debt

The Systems Engineering Approach

MLTR Framework

MLTRL - Technology Readiness Levels for Machine Learning Systems

Learn more at (Lavin et al., 2022)

Katary - Software Factory - Artificial Intelligence and Intellectual Debt

The Systems Engineering Approach

MLTR Framework
Katary - Software Factory - Artificial Intelligence and Intellectual Debt

The Systems Engineering Approach

PAISE® – Process Model for AI Systems Engineering

Learn more at (Hasterok & Stompe, 2022)

PAISE Framework
Katary - Software Factory - Artificial Intelligence and Intellectual Debt

The Systems Engineering Approach

AI Puzzle
https://xkcd.com/1838/, CC BY-NC 2.5 , via XKCD
PAISE Framework
Katary - Software Factory - Artificial Intelligence and Intellectual Debt

The Systems Engineering Approach

Results contrast with the way we work today.

"Move Fast and Break Things" (Zuckerberg, 2014)

  • Move fast and deliver working software
  • Embrace failure as a learning opportunity
  • Prioritise speed and agility
  • ...
MLTR Framework
Katary - Software Factory - Artificial Intelligence and Intellectual Debt

The Systems Engineering Approach

Results contrast with the way we work today.

"Move Fast and Break Things" (Zuckerberg, 2014)

  • Move fast and deliver working software
  • Embrace failure as a learning opportunity
  • Prioritise speed and agility
  • ...
AI Puzzle
Katary - Software Factory - Artificial Intelligence and Intellectual Debt

The Systems Engineering Approach

Inserting ML components in our software systems lowers the bar for these systems to be qualified as critical systems. Learn more at (Cabrera et al., 2025)

We need to be careful when designing, developing, deploying, and decommissioning ML-based systems.

AI Puzzle
Katary - Software Factory - Artificial Intelligence and Intellectual Debt

VibeSafe

Katary - Software Factory - Artificial Intelligence and Intellectual Debt

VibeSafe

Vibe coding is a software development practice assisted by artificial intelligence (AI) and based on chatbots (programs that simulate conversation). The software developer describes a project or task in a prompt to a large language model (LLM), which generates source code automatically. According to Wikipedia's article on Vibe coding.

PAISE Framework
Katary - Software Factory - Artificial Intelligence and Intellectual Debt

VibeSafe

Intellectual Debt: Practitioners deploy data-driven systems that work in practice, but do not fully understand their inner workings. This threatens transparency, safety, and trust, increasing risks of AI's negative social impact (Zittrain, 2022).

Katary - Software Factory - Artificial Intelligence and Intellectual Debt

VibeSafe

VibeSafe is a collection of standardized project management practices designed to promote consistent, high-quality development across projects. This is an open source project developed by Neil Lawrence and available in a GitHub repo.

PAISE Framework
Katary - Software Factory - Artificial Intelligence and Intellectual Debt

VibeSafe

Multi-agent
Two rival teams of agents - Jordan K. Terry, CC BY-SA 4.0, via Wikimedia Commons

I have been using VibeSafe for developing the DOAgent project

The goal is to develop a Python library to addreess the intellectual debt problem in Multi-Agent Systems (i.e., Agentic AI).

Katary - Software Factory - Artificial Intelligence and Intellectual Debt

Conclusions

Katary - Software Factory - Artificial Intelligence and Intellectual Debt

Conclusions

  • The scientific method provides the way to objectively assess the AI narrative. The "Technocentric" view is problematic because it ignores the real-world context. In contrast, the Systems View allows us to identify the technology limitations (e.g., Intellectual Debt).
  • The Intellectual Debt can impact our work as software developers while vibe coding. The systems engineering most basic principles are key to address this issue by following methodologies that focus more on requirements, planning, and documentation stages.
  • VibeSafe is an example of this approach. Should Katary implement its own?
Katary - Software Factory - Artificial Intelligence and Intellectual Debt

Many Thanks!

chc79@cam.ac.uk

_script: true

This script will only execute in HTML slides

_script: true