Everyday Applications

• Voice and Chat Assistants (Siri, Chat-GPT)
• Content Recommendations (Netflix, Spotify)
• Smart Home Devices
• Facial Recognition
• Language Translation
• Fraud Detection
• Personalised Marketing
• Gaming
• ...

Promises

• Early diagnosis and personalised medicine
• Drug discovery
• Climate change solutions
• Personalised learning
• Accelerating science
• Productivity boost, automation
• Smart homes, cities, etc.
• ...

Risks

• Bias and discrimination, privacy violations
• Job displacement, digital divide
• Data quality issues, system vulnerabilities
• Adversarial attacks, data breaches
• Implementation costs, market disruption
• Energy consumption, carbon footprint
• Regulatory compliance, liability issues
• ...

Risks

• Bias and discrimination, privacy violations
• Job displacement, digital divide
• Data quality issues, system vulnerabilities
• Adversarial attacks, data breaches
• Implementation costs, market disruption
• Energy consumption, carbon footprint
• Regulatory compliance, liability issues
• ...

Learning Model	Accuracy
Model v1	88%
Model v2	90%
Model v3	98%

Learning Model	Accuracy
Model v1	88%
Model v2	90%
Model v3	98%

Learning Model	Accuracy
Model v1	88%
Model v2	90%
Model v3	98%

Does Model v3 satisfy users' requirements?

Learning Model	Accuracy
Model v1	88%
Model v2	90%
Model v3	98%

Does Model v3 satisfy users' requirements?

For example:

• Users require accurate and fast responses
• The project budget is limited
• The system must be user-friendly
• Users with disabilities must be prioritised
• ...

Threat to AI applications and promises

• Mundane applications for problems we did not know we had
• Disregard for social and environmental implications
• Unrealistic expectations and hype
• Exclusion of diverse perspectives and voices
• Unsustainable technologies
• Increased inequality and digital divide
• Security and privacy concerns
• ...

"Grok referenced the debunked conspiracy theory of “white genocide” in South Africa in reply to unrelated prompts, such as questions about baseball or humorous requests to talk like a pirate. " (May, 2025)

"ChatGPT suggested a dangerous route to tourists hiking in the mountains in Poland and resulted in the hikers having to be rescued." (January, 2025)

"Steve Talley, a financial adviser in Denver, Colorado, was wrongfully arrested twice for bank robberies he did not commit, based on flawed facial recognition technology and questionable identification procedures." (September, 2014)

Learn more at the AIAAIC Repository

Latency Requirement

(< 5 secs)

Learning Model	Accuracy
Model v1	88%
Model v2	90%
Model v3	98%

Latency Requirement

(< 5 secs)

Learning Model	Accuracy	Latency
Model v1	88%	3 secs
Model v2	90%	4 secs
Model v3	98%	10 secs

Resource Limitations

(Low budget)

Learning Model	Accuracy	Latency
Model v1	88%	3 secs
Model v2	90%	4 secs
Model v3	98%	10 secs

Resource Limitations

(Low budget)

Learning Model	Accuracy	Latency	Resource Demand
Model v1	88%	3 secs	Low
Model v2	90%	4 secs	Medium
Model v3	98%	10 secs	High

Resource Limitations

(Low budget)

Learning Model	Accuracy	Latency	Resource Demand
Model v1	88%	3 secs	Low
Model v2	90%	4 secs	Medium
Model v3	98%	10 secs	High

The Ecosystem:

• Community of universities around the continent
• Inclusive training programmes
• Technical knowledge
• Students with initiative

Social Problems:

• Food security and hunger
• Poverty and inequality
• Healthcare issues (e.g., epidemics)
• Access to clean water and sanitation
• Gender inequality and violence
• Access to quality education
• Climate change and environmental problems
• ...

Important questions:

• What are the people's needs?
• Why is the problem important?
• What are the problem constraints?
• What are the important variables to consider?
• What are the relevant metrics?
• What is the data we need?
• Do we need ML?
• ...

The systems engineering approach is better equipped than the ML community to facilitate the adoption of this technology by prioritising the problems and their context before any other aspects.

Systems views: Defining the problem from different perspectives

Agility systems: Flexible architectures and solutions

Systems dynamics: Models that show systems evolution

Systems views: Defining the problem from different perspectives

Top-down analysis: Divide and conquer approach

Agility systems: Flexible architectures and solutions

Variant creation: Assessing solution alternatives

Systems dynamics: Models that show systems evolution

Problem solving cycle: Following a methodology

How does current research use the systems engineering approach to address challenges similar to the ones LLMs impose on socio-technical systems?

LLMs Applications Challenges

• Alignment and reliability
• Interpretability and accountability
• Maintainability and sustainability
• Security and privacy

LLMs Applications Challenges

• Alignment and reliability
• Interpretability and accountability
• Maintainability and sustainability
• Security and privacy

Research Question: How can we address these challenges to deploy LLMs into socio-technical systems effectively and safely?

LLMs Applications Challenges

• Alignment and reliability
• Interpretability and accountability
• Maintainability and sustainability
• Security and privacy

Research Question: How can we address these challenges to deploy LLMs into socio-technical systems effectively and safely?

Hypothesis: The Systems Engineering approach can help by prioritising the problem and its context before any solution.

LLMs Applications Challenges

• Alignment and reliability
• Interpretability and accountability
• Maintainability and sustainability
• Security and privacy

Research Question: How can we address these challenges to deploy LLMs into socio-technical systems effectively and safely?

Hypothesis: The Systems Engineering approach can help by prioritising the problem and its context before any solution.

LLMs Applications Challenges

• Alignment and reliability
• Interpretability and accountability
• Maintainability and sustainability
• Security and privacy

A survey of research works that apply systems engineering principles to address these challenges when deploying AI-based systems.

A survey of research works that apply systems engineering principles to address these challenges when deploying AI-based systems.

MLTRL - Technology Readiness Levels for Machine Learning Systems

Learn more at (Lavin et al., 2022)

PAISE® – Process Model for AI Systems Engineering

Learn more at (Hasterok & Stompe, 2022)

ACDANS – System of Systems Engineering Approach for Complex Deterministic and Nondeterministic Systems

Learn more at (Hershey, 2021)

Results contrast with the way we work today.

"Move Fast and Break Things" (Zuckerberg, 2014)

• Move fast and deliver working software
• Embrace failure as a learning opportunity
• Prioritise speed and agility
• ...

Results contrast with the way we work today.

"Move Fast and Break Things" (Zuckerberg, 2014)

• Move fast and deliver working software
• Embrace failure as a learning opportunity
• Prioritise speed and agility
• ...

Inserting ML components in our software systems lowers the bar for these systems to be qualified as critical systems. Learn more at (Cabrera et al., 2025)

We need to be careful when designing, developing, deploying, and decommissioning ML-based systems.

Overview

• The "Technocentric" View
• Context Matters
• Data Science Africa (DSA)
• ML Adoption Process
• The Problem First
• The Systems Engineering Approach

Overview

• The "Technocentric" View
• Context Matters
• Data Science Africa (DSA)
• ML Adoption Process
• The Problem First
• The Systems Engineering Approach

Next Time

• Data-orientation
• Data Science Concepts
• Data Analysis Methodology
• Access, Assess, Address