Session 1 - ML-based Systems

ML-based Systems

Christian Cabrera Jojoa

Senior Research Associate and Affiliated Lecturer

Department of Computer Science and Technology

University of Cambridge

chc79@cam.ac.uk

Session 1 - ML-based Systems

Course Structure

Session 1 - ML-based Systems

Course Structure

  • Three remote lectures combining theory and practice
  • Content will be released before each lecture
  • You can ask questions at any moment
  • Participation is key
  • Contact by email: chc79@cam.ac.uk
Session 1 - ML-based Systems

The ML Context

Session 1 - ML-based Systems

AI History

Session 1 - ML-based Systems

AI History

1940 1950 1960 1970 1980 1990 2000 2010 2020 2030
Session 1 - ML-based Systems

AI History - Inception and Early Approaches (1943 - 1969)

1940 1950 1960 1970 1980 1990 2000 2010 2020 2030 Artificial Neuron (McCulloch & Pitts, 1943) Information Theory (Shannon, 1948) Cybernetics (Wiener, 1948) Updating Rule (Hebbian, 1949) Computing Machinery and Intelligence (Turing, 1950) SNARC (Minsky, 1951) AI Term (Dartmouth Workshop, 1956) GPS (Newell & Simon, 1957) Advice Taker (McCarthy, 1958) Back-Propagation (Kelley, 1960) Perceptrons (Rosenblatt, 1962) ELIZA (MIT, 1966) The DENDRAL (Buchanan, 1969)
Session 1 - ML-based Systems

Early AI Approaches (1943-1969) - Searching Algorithms

Problem Definition

Formal Definition

Computational Representation

An agent must find a way to reach a goal in its environment. But, the next step is not obvious.

Shortest path problem
Shortest path problem: https://en.wikipedia.org/wiki/Shortest_path_problem

A search problem is defined by a set of states, an initial state, a set of goal states, and a set of actions or transitions between states.

: set of states
: set of actions
: initial state
: set of goal states

A search problem is typically modelled using data structures such as graphs, adjacency matrices, queues and stacks.

Adjacency matrix for shortest path graph
Adjacency matrix
Session 1 - ML-based Systems

Early AI Approaches (1943-1969) - AI Perception

Session 1 - ML-based Systems

Early AI Approaches (1943-1969) - AI Perception

"Within ten years a digital computer will be the world's chess champion." (Simon & Newell, 1958)


"Machines will be capable, within twenty years, of doing any work a man can do." (Simon, 1965)


"Within a generation... the problem of creating 'artificial intelligence' will substantially be solved." (Minsky, 1967)


"In from three to eight years we will have a machine with the general intelligence of an average human being." (Minsky, 1970)

You'll own slaves by 1965 (1957)
You'll own "slaves" by 1965 (1957) - https://medium.com/@theo/do-we-need-robot-rights-in-the-age-of-artificial-intelligence-690b9951bae0
Session 1 - ML-based Systems

Early AI Approaches (1943-1969) - AI Perception

"Within ten years a digital computer will be the world's chess champion." (Simon & Newell, 1958)


"Machines will be capable, within twenty years, of doing any work a man can do." (Simon, 1965)


"Within a generation... the problem of creating 'artificial intelligence' will substantially be solved." (Minsky, 1967)


"In from three to eight years we will have a machine with the general intelligence of an average human being." (Minsky, 1970)

Machines that Think Held Evil (1960)
Machines that Think Held Evil (1960) - https://newsletter.pessimistsarchive.org/p/the-original-ai-doomer-dr-norbert
Session 1 - ML-based Systems

Early AI Approaches (1943-1969) - AI Perception

"Within ten years a digital computer will be the world's chess champion." (Simon & Newell, 1958)


"Machines will be capable, within twenty years, of doing any work a man can do." (Simon, 1965)


"Within a generation... the problem of creating 'artificial intelligence' will substantially be solved." (Minsky, 1967)


"In from three to eight years we will have a machine with the general intelligence of an average human being." (Minsky, 1970)

Automation Concerns (1960)
Automation Concerns (1960) - https://newsletter.pessimistsarchive.org/p/the-original-ai-doomer-dr-norbert
Session 1 - ML-based Systems

Early AI Approaches (1943-1969) - AI Perception

"Within ten years a digital computer will be the world's chess champion." (Simon & Newell, 1958)


"Machines will be capable, within twenty years, of doing any work a man can do." (Simon, 1965)


"Within a generation... the problem of creating 'artificial intelligence' will substantially be solved." (Minsky, 1967)


"In from three to eight years we will have a machine with the general intelligence of an average human being." (Minsky, 1970)

IBM Response (1960)
IBM Response (1960) - https://newsletter.pessimistsarchive.org/p/the-original-ai-doomer-dr-norbert
Session 1 - ML-based Systems

Early AI Approaches (1943-1969) - AI Limitations

Session 1 - ML-based Systems

Early AI Approaches (1943-1969) - AI Limitations

Computing power, algorithms, and data were insufficient to solve real-world problems.


Combinatorial explosion


Easy tasks for humans are difficult for AI (Moravec's Paradox)

Session 1 - ML-based Systems

Early AI Approaches (1943-1969) - AI Limitations

Maze navigation search problem

Computing power, algorithms, and data were insufficient to solve real-world problems.


Combinatorial explosion


Easy tasks for humans are difficult for AI (Moravec's Paradox)

Session 1 - ML-based Systems

Early AI Approaches (1943-1969) - AI Limitations

Combinatorial explosion diagram

Computing power, algorithms, and data were insufficient to solve real-world problems.


Combinatorial explosion


Easy tasks for humans are difficult for AI (Moravec's Paradox)

Session 1 - ML-based Systems

Early AI Approaches (1943-1969) - AI Limitations

Growth comparison diagram

Computing power, algorithms, and data were insufficient to solve real-world problems.


Combinatorial explosion


Easy tasks for humans are difficult for AI (Moravec's Paradox)

Session 1 - ML-based Systems

AI History - First Winter (1974 - 1980)

1940 1950 1960 1970 1980 1990 2000 2010 2020 2030 First AI Winter (1974-1980) Artificial Neuron (McCulloch & Pitts, 1943) Information Theory (Shannon, 1948) Cybernetics (Wiener, 1948) Updating Rule (Hebbian, 1949) Computing Machinery and Intelligence (Turing, 1950) SNARC (Minsky, 1951) AI Term (Dartmouth Workshop, 1956) GPS (Newell & Simon, 1957) Advice Taker (McCarthy, 1958) Back-Propagation (Kelley, 1960) Perceptrons (Rosenblatt, 1962) ELIZA (MIT, 1966) ALPAC Report (USA, 1966) The DENDRAL (Buchanan, 1969) Perceptrons Book (Minsky & Papert, 1969) Lighthill Report (UK, 1973)
Session 1 - ML-based Systems

AI History - Expert Systems (1969 - 1986)

1940 1950 1960 1970 1980 1990 2000 2010 2020 2030 First AI Winter (1974-1980) Artificial Neuron (McCulloch & Pitts, 1943) Information Theory (Shannon, 1948) Cybernetics (Wiener, 1948) Updating Rule (Hebbian, 1949) Computing Machinery and Intelligence (Turing, 1950) SNARC (Minsky, 1951) AI Term (Dartmouth Workshop, 1956) GPS (Newell & Simon, 1957) Advice Taker (McCarthy, 1958) Back-Propagation (Kelley, 1960) Perceptrons (Rosenblatt, 1962) ELIZA (MIT, 1966) ALPAC Report (USA, 1966) The DENDRAL (Buchanan, 1969) Perceptrons Book (Minsky & Papert, 1969) PROLOG (1972) MYCIN (Stanford, 1972) Lighthill Report (UK, 1973) FRAMES (1975) Hopfield net (1982) R1 (McDermott, 1982) Parallel Distributed Processing (Rumelhart & McClelland, 1986)
Session 1 - ML-based Systems

Expert Systems (1969-1986) - Knowledge-Based Systems

Problem Definition

Formal Definition

Computational Representation

Agents are limited because of the problem's complexity. They should leverage human knowledge and emulate human reasoning.

Data, Information, and Knowledge
Data, Information, and Knowledge (Per Liew, 2007)

An expert system is defined by a knowledge base, inference engine, and user interface that work together to apply domain expertise to specific problems.

: Knowledge Base
: Inference Engine
: User Interface

Knowledge is typically modelled using subject, object, predicate semantic triple model.

Semantic Triple Model
Basic Semantic Triple

Rules are if statements.

Session 1 - ML-based Systems

Expert Systems (1969-1986) - Relative Success

Session 1 - ML-based Systems

Expert Systems (1969-1986) - Relative Success

DENDRAL (1960s)

Identified unknown organic molecules using knowledge of chemistry


MYCIN (Early 1970s)

Supported bacterial infections diagnosis and treatment


XCON/R1 (1982)

eXpert CONfigurer - Automated the configuration of VAX computer systems (successful deployment)

PUFF (1982)

Interpreted pulmonary function test results to diagnose lung disorders


PROSPECTOR (1986)

An expert system for mineral exploration


DEEP BLUE (1997)

An expert system that defeated a chess world champion

Session 1 - ML-based Systems

Expert Systems (1969-1986) - AI Perception

Session 1 - ML-based Systems

Expert Systems (1969-1986) - AI Perception

"In medicine, management, and the military — indeed in most of the world's work — the daily tasks are those requiring symbolic reasoning with detailed professional knowledge." (Feigenbaum, 1982)


"Commercialising Artificial Intelligence." (The New York Times, 1982)


"Gains are Slow for Artificial Intelligence Industry." (The New York Times, 1987)


"New expert systems companies were being formed at a rate of what seemed like one a week. " (Hart, 2021)


Session 1 - ML-based Systems

Expert Systems (1969-1986) - AI Perception

Artificial Intelligence (1981)
Issues with knowledge libraries (Robersts, 1981) - https://microship.com/artificial-intelligence-byte/

"In medicine, management, and the military — indeed in most of the world's work — the daily tasks are those requiring symbolic reasoning with detailed professional knowledge." (Feigenbaum, 1982)


"Commercialising Artificial Intelligence." (The New York Times, 1982)


"Gains are Slow for Artificial Intelligence Industry." (The New York Times, 1987)


"New expert systems companies were being formed at a rate of what seemed like one a week. " (Hart, 2021)


Session 1 - ML-based Systems

Expert Systems (1969-1986) - AI Perception

Intelligence is more than experts (1985)
Intelligence is More than Experts (The Sydney Morning Herald, 1985)

"In medicine, management, and the military — indeed in most of the world's work — the daily tasks are those requiring symbolic reasoning with detailed professional knowledge." (Feigenbaum, 1982)


"Commercialising Artificial Intelligence." (The New York Times, 1982)


"Gains are Slow for Artificial Intelligence Industry." (The New York Times, 1987)


"New expert systems companies were being formed at a rate of what seemed like one a week. " (Hart, 2021)


Session 1 - ML-based Systems

Expert Systems (1969-1986) - AI Perception

AI just could be a smart buy (1987)
AI just could be a smart buy (1987)

"In medicine, management, and the military — indeed in most of the world's work — the daily tasks are those requiring symbolic reasoning with detailed professional knowledge." (Feigenbaum, 1982)


"Commercialising Artificial Intelligence." (The New York Times, 1982)


"Gains are Slow for Artificial Intelligence Industry." (The New York Times, 1987)


"New expert systems companies were being formed at a rate of what seemed like one a week. " (Hart, 2021)


Session 1 - ML-based Systems

Expert Systems (1969-1986) - AI Perception

Deep Blue defeats Kasparov (1997)
Deep Blue defeats Kasparov (Los Angeles Times, 1997)

"In medicine, management, and the military — indeed in most of the world's work — the daily tasks are those requiring symbolic reasoning with detailed professional knowledge." (Feigenbaum, 1982)


"Commercialising Artificial Intelligence." (The New York Times, 1982)


"Gains are Slow for Artificial Intelligence Industry." (The New York Times, 1987)


"New expert systems companies were being formed at a rate of what seemed like one a week. " (Hart, 2021)


Session 1 - ML-based Systems

Expert Systems (1969-1986) - AI Limitations

Session 1 - ML-based Systems

Expert Systems (1969-1986) - AI Limitations

Knowledge Complexity
The Tree of Knowledge System: Gregg Henriques, CC BY-SA 4.0 , via Wikimedia Commons

Extracting knowledge from human experts and encoding it into rules was difficult, time-consuming, and expensive


Systems could not reason beyond their pre-programmed knowledge and failed when confronted with unexpected situations


Adding more rules often led to rule interaction problems and combinatorial explosion


Updating knowledge bases as domains evolved required significant effort

Session 1 - ML-based Systems

AI History - Second AI Winter (1987 - 1994)

1940 1950 1960 1970 1980 1990 2000 2010 2020 2030 First AI Winter (1974-1980) Second AI Winter (1987-1994) Artificial Neuron (McCulloch & Pitts, 1943) Information Theory (Shannon, 1948) Cybernetics (Wiener, 1948) Updating Rule (Hebbian, 1949) Computing Machinery and Intelligence (Turing, 1950) SNARC (Minsky, 1951) AI Term (Dartmouth Workshop, 1956) GPS (Newell & Simon, 1957) Advice Taker (McCarthy, 1958) Back-Propagation (Kelley, 1960) Perceptrons (Rosenblatt, 1962) ELIZA (MIT, 1966) ALPAC Report (USA, 1966) The DENDRAL (Buchanan, 1969) Perceptrons Book (Minsky & Papert, 1969) PROLOG (1972) MYCIN (Stanford, 1972) Lighthill Report (UK, 1973) FRAMES (1975) Hopfield net (1982) R1 (McDermott, 1982) Parallel Distributed Processing (Rumelhart & McClelland, 1986)
Session 1 - ML-based Systems

AI History - Learning from Examples (1987 - present)

1940 1950 1960 1970 1980 1990 2000 2010 2020 2030 Artificial Neuron (McCulloch & Pitts, 1943) First AI Winter (1974-1980) Second AI Winter (1987-1994) Artificial Neuron (McCulloch & Pitts, 1943) Information Theory (Shannon, 1948) Cybernetics (Wiener, 1948) Updating Rule (Hebbian, 1949) Computing Machinery and Intelligence (Turing, 1950) SNARC (Minsky, 1951) AI Term (Dartmouth Workshop, 1956) GPS (Newell & Simon, 1957) Advice Taker (McCarthy, 1958) Back-Propagation (Kelley, 1960) Perceptrons (Rosenblatt, 1962) ELIZA (MIT, 1966) ALPAC Report (USA, 1966) The DENDRAL (Buchanan, 1969) Perceptrons Book (Minsky & Papert, 1969) PROLOG (1972) MYCIN (Stanford, 1972) Lighthill Report (UK, 1973) FRAMES (1975) Hopfield net (1982) R1 (McDermott, 1982) Parallel Distributed Processing (Rumelhart & McClelland, 1986) Bayesian Networks (Pearls, 1988) Reinforcement Learning (Sutton, 1988) Image Recognition (LeCun et al., 1990) Deep Blue beats Kasparov (IBM, 1997)
Session 1 - ML-based Systems

Learning from Examples (1987 - present) - Data-Driven Algorithms

Problem Definition

Formal Definition

Computational Representation

Session 1 - ML-based Systems

Learning from Examples (1987 - present) - Data-Driven Algorithms

Problem Definition

Formal Definition

Computational Representation

Agents cannot be fully pre-programmed. Agent must learn from examples to perform a task.

Learning from data
Learning from examples: classification problem
Session 1 - ML-based Systems

Learning from Examples (1987 - present) - Data-Driven Algorithms

Problem Definition

Formal Definition

Computational Representation

Agents cannot be fully pre-programmed. Agent must learn from examples to perform a task.

Learning from data
Learning from examples: classification problem

Learning from examples requires a function that approximates patterns in data, using a learning algorithm, and evaluation criteria.

: dataset of examples or pairs
: hypothesis space of possible functions
: loss/reward function measuring success
: algorithm to search through

Session 1 - ML-based Systems

Learning from Examples (1987 - present) - Data-Driven Algorithms

Problem Definition

Formal Definition

Computational Representation

Agents cannot be fully pre-programmed. Agent must learn from examples to perform a task.

Learning from data
Learning from examples: classification problem

Learning from examples requires a function that approximates patterns in data, using a learning algorithm, and evaluation criteria.

: dataset of examples or pairs
: hypothesis space of possible functions
: loss/reward function measuring success
: algorithm to search through

Learning algorithms use different computational representations during the learning and inference: dataframes, tuples, trees, graphs, matrices, etc.

Session 1 - ML-based Systems

Learning from Examples (1987 - present) - Data-Driven Algorithms

Algorithms
Session 1 - ML-based Systems

Learning from Examples (1987 - present) - Data-Driven Algorithms

Algorithms
Supervised vs Non-supervised
Supervised vs Non-supervised: Balkiss.hamad, CC BY-SA 4.0 , via Wikimedia Commons
Session 1 - ML-based Systems

Learning from Examples (1987 - present) - Data-Driven Algorithms

Algorithms
Supervised vs Non-supervised
Supervised vs Non-supervised: Balkiss.hamad, CC BY-SA 4.0 , via Wikimedia Commons
Reinforcement Learning
Reinforcement Learning
Session 1 - ML-based Systems

Learning from Examples (1987 - present) - Data-Driven Algorithms

Models
Session 1 - ML-based Systems

Learning from Examples (1987 - present) - Data-Driven Algorithms

Models
Linear Regression Model
Linear regression model: Sewaqu, Public domain, via Wikimedia Commons, via Wikimedia Commons
Session 1 - ML-based Systems

Learning from Examples (1987 - present) - Data-Driven Algorithms

Models
Linear Regression Model
Linear regression model: Sewaqu, Public domain, via Wikimedia Commons, via Wikimedia Commons
Neural Network Model
Neural Network Model: Glosser.ca, CC BY-SA 3.0 , via Wikimedia Commons
Session 1 - ML-based Systems

AI History - Big Data (2000s)

1940 1950 1960 1970 1980 1990 2000 2010 2020 2030 Artificial Neuron (McCulloch & Pitts, 1943) First AI Winter (1974-1980) Second AI Winter (1987-1994) Big Data (2000-2012) Artificial Neuron (McCulloch & Pitts, 1943) Information Theory (Shannon, 1948) Cybernetics (Wiener, 1948) Updating Rule (Hebbian, 1949) Computing Machinery and Intelligence (Turing, 1950) SNARC (Minsky, 1951) AI Term (Dartmouth Workshop, 1956) GPS (Newell & Simon, 1957) Advice Taker (McCarthy, 1958) Back-Propagation (Kelley, 1960) Perceptrons (Rosenblatt, 1962) ELIZA (MIT, 1966) ALPAC Report (USA, 1966) The DENDRAL (Buchanan, 1969) Perceptrons Book (Minsky & Papert, 1969) PROLOG (1972) MYCIN (Stanford, 1972) Lighthill Report (UK, 1973) FRAMES (1975) Hopfield net (1982) R1 (McDermott, 1982) Parallel Distributed Processing (Rumelhart & McClelland, 1986) Bayesian Networks (Pearls, 1988) Reinforcement Learning (Sutton, 1988) Image Recognition (LeCun et al., 1990) Deep Blue beats Kasparov (IBM, 1997)
Session 1 - ML-based Systems

AI History - Big Data (2000s)

Session 1 - ML-based Systems

AI History - Big Data (2000s)

Big Data
Big Data: Camelia.boban, CC BY-SA 3.0 , via Wikimedia Commons
Session 1 - ML-based Systems

AI History - Big Data (2000s)

Large and complex datasets that improve models' statistical power

  • Social Networks
  • Mobile Computing
  • Internet of Things
  • ...

Big Data
Big Data: Camelia.boban, CC BY-SA 3.0 , via Wikimedia Commons
Session 1 - ML-based Systems

AI History - Big Data (2000s)

Moore's Law
Session 1 - ML-based Systems

AI History - Machine Learning Age (2001 - present)

1940 1950 1960 1970 1980 1990 2000 2010 2020 2030 First AI Winter (1974-1980) Second AI Winter (1987-1994) Big Data (2000-2012) Artificial Neuron (McCulloch & Pitts, 1943) Information Theory (Shannon, 1948) Cybernetics (Wiener, 1948) Updating Rule (Hebbian, 1949) Computing Machinery and Intelligence (Turing, 1950) SNARC (Minsky, 1951) AI Term (Dartmouth Workshop, 1956) GPS (Newell & Simon, 1957) Advice Taker (McCarthy, 1958) Back-Propagation (Kelley, 1960) Perceptrons (Rosenblatt, 1962) ELIZA (MIT, 1966) ALPAC Report (USA, 1966) The DENDRAL (Buchanan, 1969) Perceptrons Book (Minsky & Papert, 1969) PROLOG (1972) MYCIN (Stanford, 1972) Lighthill Report (UK, 1973) FRAMES (1975) Hopfield net (1982) R1 (McDermott, 1982) Parallel Distributed Processing (Rumelhart & McClelland, 1986) Bayesian Networks (Pearls, 1988) Reinforcement Learning (Sutton, 1988) Image Recognition (LeCun et al., 1990) Deep Blue beats Kasparov (IBM, 1997) Deep Learning (Hinton, 2006) Watson wins Jeopardy (2011) AlexNet (Krizhevsky, 2012) GANs (Goodfellow, 2014) AlphaGo beats Lee Sedol (DeepMind, 2016) Transformer (Vaswani, 2017) AlphaFold (DeepMind, 2018) GPT-1 (OpenAI, 2020) BERT (Google, 2019) Chinchilla (DeepMind, 2022) ChatGPT (OpenAI, 2022) LLaMA (Meta AI, 2023) Claude 2 (Anthropic, 2023) phi-3 (Microsoft, 2024) Gemini 1.5 (Google DeepMind, 2024) Qwen3 (Alibaba, 2025) R1 (DeepSeek) 2025)
Session 1 - ML-based Systems

ML Today

Session 1 - ML-based Systems

AI Today - Applications

Session 1 - ML-based Systems

AI Today - Applications

Everyday Applications

  • Voice and Chat Assistants (Siri, Chat-GPT)
  • Content Recommendations (Netflix, Spotify)
  • Smart Home Devices
  • Facial Recognition
  • Language Translation
  • Fraud Detection
  • Personalised Marketing
  • Gaming
  • ...
Applications
Session 1 - ML-based Systems

AI Today - Promises

Session 1 - ML-based Systems

AI Today - Promises

Promises

  • Early diagnosis and personalised medicine
  • Drug discovery
  • Climate change solutions
  • Personalised learning
  • Accelerating science
  • Productivity boost, automation
  • Smart homes, cities, etc.
  • ...
Protein
Protein Folding: BQUB23-Anialet, CC BY-SA 4.0 , via Wikimedia Commons
Session 1 - ML-based Systems

AI Today - Risks

Session 1 - ML-based Systems

AI Today - Risks

Risks
Designed by stories / Freepik

Risks

  • Bias and discrimination, privacy violations
  • Job displacement, digital divide
  • Data quality issues, system vulnerabilities
  • Adversarial attacks, data breaches
  • Implementation costs, market disruption
  • Energy consumption, carbon footprint
  • Regulatory compliance, liability issues
  • ...
Session 1 - ML-based Systems

AI Today - Risks

Risks

Risks

  • Bias and discrimination, privacy violations
  • Job displacement, digital divide
  • Data quality issues, system vulnerabilities
  • Adversarial attacks, data breaches
  • Implementation costs, market disruption
  • Energy consumption, carbon footprint
  • Regulatory compliance, liability issues
  • ...
Session 1 - ML-based Systems

ML Perception

Session 1 - ML-based Systems

ML Perception

Perception
Session 1 - ML-based Systems

ML Perception

Perception
Session 1 - ML-based Systems

ML Perception

Perception
Session 1 - ML-based Systems

ML Perception

Perception
Session 1 - ML-based Systems

ML Perception

"We should be able to do 90 percent of miles driven [autonomously] within three years." (Musk, 2013)


"From a technology standpoint, Tesla will have a car that can do full autonomy in about three years, maybe a bit sooner." (Musk, 2015)


"I feel very confident predicting that there will be autonomous robotaxis from Tesla next year." (Musk, 2019)

...

Session 1 - ML-based Systems

ML Perception

"We should be able to do 90 percent of miles driven [autonomously] within three years." (Musk, 2013)


"From a technology standpoint, Tesla will have a car that can do full autonomy in about three years, maybe a bit sooner." (Musk, 2015)


"I feel very confident predicting that there will be autonomous robotaxis from Tesla next year." (Musk, 2019)

...

Musk
See full list of predictions: https://en.wikipedia.org/wiki/List_of_predictions_for_autonomous_Tesla_vehicles_by_Elon_Musk
Session 1 - ML-based Systems

ML Perception

"We should be able to do 90 percent of miles driven [autonomously] within three years." (Musk, 2013)


"From a technology standpoint, Tesla will have a car that can do full autonomy in about three years, maybe a bit sooner." (Musk, 2015)


"I feel very confident predicting that there will be autonomous robotaxis from Tesla next year." (Musk, 2019)

...

"I predict that there will be millions of Teslas operating fully autonomously in the second half of next year." (Musk, 2025)

Musk
See full list of predictions: https://en.wikipedia.org/wiki/List_of_predictions_for_autonomous_Tesla_vehicles_by_Elon_Musk
Session 1 - ML-based Systems

ML Perception

Geoffrey Hinton
NOBEL Prizes Ceremony 2024: Arthur Petron, CC BY-SA 4.0 , via Wikimedia Commons
Session 1 - ML-based Systems

ML Perception

"We should stop training radiologists now. It’s just completely obvious that within five years, deep learning is going to do better than radiologists." (Hinton, 2016)

Geoffrey Hinton
NOBEL Prizes Ceremony 2024: Arthur Petron, CC BY-SA 4.0 , via Wikimedia Commons
Session 1 - ML-based Systems

ML Perception

"We should stop training radiologists now. It’s just completely obvious that within five years, deep learning is going to do better than radiologists." (Hinton, 2016)

Radiology Crisis
Radiology Crisis - Independent: https://www.independent.co.uk/news/health/cancer-delays-nhs-staff-shortages-b2561385.html
Session 1 - ML-based Systems

ML Perception

The Economist
The economics of superintelligence - The Economist (26th July - 8th August 2025)
Session 1 - ML-based Systems

ML Perception

The Economist
The economics of superintelligence - The Economist (26th July - 8th August 2025)

"AI will be capable of generating novel insights next year." (Altman, 2025)


"Self-improving AI will create a super intelligence." (Musk, 2025)


"When I look at the data, I see many trend lines up to 2027." (Clark, 2025)


"There is a 10-20% chance that the technology will end in human extinction." (Hinton, 2025)


"Our relationship with future AI systems is that we are going to be their boss." (LeCun, 2025)

Session 1 - ML-based Systems

ML Perception

"Machines will be capable, within twenty years, of doing any work a man can do." (Simon, 1965)


"In from three to eight years we will have a machine with the general intelligence of an average human being." (Minsky, 1970)


"In medicine, management, and the military — indeed in most of the world's work — the daily tasks are those requiring symbolic reasoning with detailed professional knowledge." (Feigenbaum, 1982)

"AI will be capable of generating novel insights next year." (Altman, 2025)


"Self-improving AI will create a super intelligence." (Musk, 2025)


"When I look at the data, I see many trend lines up to 2027." (Clark, 2025)


"There is a 10-20% chance that the technology will end in human extinction." (Hinton, 2025)


"Our relationship with future AI systems is that we are going to be their boss." (LeCun, 2025)

Session 1 - ML-based Systems

ML Perception

You'll own slaves by 1965 (1957)
You'll own "slaves" by 1965 (1957) - https://medium.com/@theo/do-we-need-robot-rights-in-the-age-of-artificial-intelligence-690b9951bae0

"AI will be capable of generating novel insights next year." (Altman, 2025)


"Self-improving AI will create a super intelligence." (Musk, 2025)


"When I look at the data, I see many trend lines up to 2027." (Clark, 2025)


"There is a 10-20% chance that the technology will end in human extinction." (Hinton, 2025)


"Our relationship with future AI systems is that we are going to be their boss." (LeCun, 2025)

Session 1 - ML-based Systems

ML Perception

AI just could be a smart buy (1987)
AI just could be a smart buy (1987)
The Economist
The economics of superintelligence - The Economist (26th July - 8th August 2025)
Session 1 - ML-based Systems

ML Perception

Demis Hassabis
NOBEL Prizes Ceremony 2024: Arthur Petron, CC BY-SA 4.0 , via Wikimedia Commons
Protein
Protein Folding: BQUB23-Anialet, CC BY-SA 4.0 , via Wikimedia Commons
Session 1 - ML-based Systems

ML Perception

Demis Hassabis
NOBEL Prizes Ceremony 2024: Arthur Petron, CC BY-SA 4.0 , via Wikimedia Commons

"... step 1: solving intelligence, step 2: use it to solve everything else..." (Hassabis, 2025)


"Artificial General Intelligence (AGI) will emerge in the next five or 10 years." (Hassabis, 2025)

Session 1 - ML-based Systems

ML Perception

Demis Hassabis
NOBEL Prizes Ceremony 2024: Arthur Petron, CC BY-SA 4.0 , via Wikimedia Commons

Machine Learning Problem Requirements

1. Massive combinatorial search space


2. Clear objective function (metric) to optimise against


3. Lots of data and/or accurate and efficient simulators

Session 1 - ML-based Systems

ML Perception

The Economist
The economics of superintelligence - The Economist (26th July - 8th August 2025)

Machine Learning Problem Requirements

1. Massive combinatorial search space


2. Clear objective function (metric) to optimise against


3. Lots of data and/or accurate and efficient simulators

Session 1 - ML-based Systems

ML Perception

Social Problems
Social Problems Metrics?: Jusezam, CC BY-SA 3.0 , via Wikimedia Commons

Machine Learning Problem Requirements

1. Massive combinatorial search space


2. Clear objective function (metric) to optimise against


3. Lots of data and/or accurate and efficient simulators

Session 1 - ML-based Systems

ML Perception

AI Models Collapse
AI Models Collapse: https://www.nature.com/articles/s41586-024-07566-y

Machine Learning Problem Requirements

1. Massive combinatorial search space


2. Clear objective function (metric) to optimise against


3. Lots of data and/or accurate and efficient simulators

Session 1 - ML-based Systems

Scientific Approach

Critical Thinking
Critical Thinking (Designed by freepik.com)
Session 1 - ML-based Systems

Scientific Approach

Critical Thinking
Critical Thinking (Designed by freepik.com)

"It seems to me what is called for is an exquisite balance between two conflicting needs: the most skeptical scrutiny of all hypotheses that are served up to us and at the same time a great openness to new ideas. Obviously those two modes of thought are in some tension. But if you are able to exercise only one of these modes, whichever one it is, you’re in deep trouble. (The Burden of Skepticism, Sagan, 1987)

Session 1 - ML-based Systems

ML Definition

Session 1 - ML-based Systems

ML Definition

Session 1 - ML-based Systems

ML Definition

Session 1 - ML-based Systems

ML Definition

Our definition before...


Machine Learning Definition
Session 1 - ML-based Systems

ML Definition

Session 1 - ML-based Systems

ML Definition

Our ML projects must have a purpose...


People
Session 1 - ML-based Systems

The "Technocentric" View

Session 1 - ML-based Systems

The "Technocentric" View

Session 1 - ML-based Systems

The "Technocentric" View

AI Puzzle
Session 1 - ML-based Systems

The "Technocentric" View

Single Model
AI Puzzle
Session 1 - ML-based Systems

The "Technocentric" View

Single Model
ML System?
https://xkcd.com/1838/, CC BY-NC 2.5 , via XKCD
Session 1 - ML-based Systems

The "Technocentric" View

Single Model
Learning Model Accuracy
Model v1 88%
Model v2 90%
Model v3 98%
Session 1 - ML-based Systems

The "Technocentric" View

Single Model
Learning Model Accuracy
Model v1 88%
Model v2 90%
Model v3 98%
Session 1 - ML-based Systems

The "Technocentric" View

Single Model
Learning Model Accuracy
Model v1 88%
Model v2 90%
Model v3 98%

Does Model v3 satisfy users' requirements?

Session 1 - ML-based Systems

The "Technocentric" View

Single Model
Learning Model Accuracy
Model v1 88%
Model v2 90%
Model v3 98%

Does Model v3 satisfy users' requirements?

For example:

  • Users require accurate and fast responses
  • The project budget is limited
  • The system must be user-friendly
  • Users with disabilities must be prioritised
  • ...
Session 1 - ML-based Systems

The "Technocentric" View

Single Model

Threat to AI applications and promises


  • Mundane applications for problems we did not know we had
  • Disregard for social and environmental implications
  • Unrealistic expectations and hype
  • Exclusion of diverse perspectives and voices
  • Unsustainable technologies
  • Increased inequality and digital divide
  • Security and privacy concerns
  • ...
Session 1 - ML-based Systems

The "Technocentric" View

Single Model

"Grok referenced the debunked conspiracy theory of “white genocide” in South Africa in reply to unrelated prompts, such as questions about baseball or humorous requests to talk like a pirate. " (May, 2025)


"ChatGPT suggested a dangerous route to tourists hiking in the mountains in Poland and resulted in the hikers having to be rescued." (January, 2025)


"Steve Talley, a financial adviser in Denver, Colorado, was wrongfully arrested twice for bank robberies he did not commit, based on flawed facial recognition technology and questionable identification procedures." (September, 2014)

Learn more at the AIAAIC Repository

Session 1 - ML-based Systems

Context Matters

Session 1 - ML-based Systems

Context Matters

Session 1 - ML-based Systems

Context Matters

AI Puzzle
Session 1 - ML-based Systems

Context Matters

AI Puzzle
Session 1 - ML-based Systems

Context Matters

AI Puzzle
Session 1 - ML-based Systems

Context Matters

AI Puzzle
Session 1 - ML-based Systems

Context Matters

AI Puzzle
Learning Model Accuracy
Model v1 88%
Model v2 90%
Model v3 98%
Session 1 - ML-based Systems

Context Matters

AI Puzzle

Latency Requirement

(< 5 secs)

Learning Model Accuracy
Model v1 88%
Model v2 90%
Model v3 98%
Session 1 - ML-based Systems

Context Matters

AI Puzzle

Latency Requirement

(< 5 secs)

Learning Model Accuracy Latency
Model v1 88% 3 secs
Model v2 90% 4 secs
Model v3 98% 10 secs
Session 1 - ML-based Systems

Context Matters

AI Puzzle

Resource Limitations

(Low budget)

Learning Model Accuracy Latency
Model v1 88% 3 secs
Model v2 90% 4 secs
Model v3 98% 10 secs
Session 1 - ML-based Systems

Context Matters

AI Puzzle

Resource Limitations

(Low budget)

Learning Model Accuracy Latency Resource Demand
Model v1 88% 3 secs Low
Model v2 90% 4 secs Medium
Model v3 98% 10 secs High
Session 1 - ML-based Systems

Context Matters

AI Puzzle

Resource Limitations

(Low budget)

Learning Model Accuracy Latency Resource Demand
Model v1 88% 3 secs Low
Model v2 90% 4 secs Medium
Model v3 98% 10 secs High
Session 1 - ML-based Systems

Context Matters

Context People
Session 1 - ML-based Systems

Example 1

Session 1 - ML-based Systems

Data Science Africa (DSA)

Session 1 - ML-based Systems

Data Science Africa (DSA)

Session 1 - ML-based Systems

Data Science Africa (DSA)

The Ecosystem:

  • Community of universities around the continent
  • Inclusive training programmes
  • Technical knowledge
  • Students with initiative
Session 1 - ML-based Systems

Data Science Africa (DSA)

Social Problems:

  • Food security and hunger
  • Poverty and inequality
  • Healthcare issues (e.g., epidemics)
  • Access to clean water and sanitation
  • Gender inequality and violence
  • Access to quality education
  • Climate change and environmental problems
  • ...
Session 1 - ML-based Systems

Data Science Africa (DSA)

Ewaso Nyiro River
Ewaso Nyiro River - Kenya: Marc Samsom, CC BY 2.0 , via Wikimedia Commons
Session 1 - ML-based Systems

Data Science Africa (DSA)

Ewaso Nyiro River
Ewaso Nyiro River - Kenya: Marc Samsom, CC BY 2.0 , via Wikimedia Commons
Water Level Monitoring
Water Level Monitoring System at DeKUT (Kabi & Maina, 2021)
Session 1 - ML-based Systems

Data Science Africa (DSA)

Water Level Monitoring System Architecture
Water Level Monitoring System Architecture
Session 1 - ML-based Systems

Data Science Africa (DSA)

Water Level Monitoring System Architecture
Water Level Monitoring System Architecture
Session 1 - ML-based Systems

Example 2

Session 1 - ML-based Systems

Service Placement Problem

Service Placement Problem

Dynamic Service Placement in Edge Computing


Session 1 - ML-based Systems

Service Placement Problem

Service Placement Problem

Dynamic Service Placement in Edge Computing


  • Edge servers are located close to end users, allowing for local data processing.
  • Services run on edge servers, which have limited resources.
  • The challenge is to determine the optimal allocation of services and edge servers to minimize latency while considering resource constraints.
  • This challenge is referred to as the Service Placement Problem.
Session 1 - ML-based Systems

Service Placement Problem

Service Placement Problem

Dynamic Service Placement in Edge Computing


Objective Functions:


Subject to:

Session 1 - ML-based Systems

Service Placement Problem

Pareto

Dynamic Service Placement in Edge Computing


Objective Functions:


Subject to:

Session 1 - ML-based Systems

Service Placement Problem

Ant-Colony Optimisation

Dynamic Service Placement in Edge Computing


Objective Functions:


Subject to:

Session 1 - ML-based Systems

Service Placement Problem

Ant-Colony Optimisation

Ant Colony Optimization Algorithm



Where: is the probability of moving from node to node , is the pheromone level on edge at time , is the heuristic information (e.g., inverse of distance), and are parameters to control the influence of pheromone and heuristic information, is the pheromone evaporation rate, is change in pheromone level.

Session 1 - ML-based Systems

Service Placement Problem

Ant-Colony Optimisation
Session 1 - ML-based Systems

Service Placement Problem

Ant-Colony Optimisation
Session 1 - ML-based Systems

Service Placement Problem

Ant-Colony Optimisation
Session 1 - ML-based Systems

Service Placement Problem

Ant-Colony Optimisation
Session 1 - ML-based Systems

Service Placement Problem

Ant-Colony Optimisation
Session 1 - ML-based Systems

Service Placement Problem

Ant-Colony Optimisation
Session 1 - ML-based Systems

Service Placement Problem

Ant-Colony Optimisation
Session 1 - ML-based Systems

Service Placement Problem

Ant-Colony Optimisation
Session 1 - ML-based Systems

Service Placement Problem

Ant-Colony Optimisation
Session 1 - ML-based Systems

Service Placement Problem

Ant-Colony Optimisation
Session 1 - ML-based Systems

Service Placement Problem

Ant-Colony Optimisation
Session 1 - ML-based Systems

Service Placement Problem

Ant-Colony Optimisation
Session 1 - ML-based Systems

Service Placement Problem

Ant-Colony Optimisation
Session 1 - ML-based Systems

Service Placement Problem

Ant-Colony Optimisation
Session 1 - ML-based Systems

Service Placement Problem

Ant-Colony Optimisation
Session 1 - ML-based Systems

Service Placement Problem

Ant-Colony Optimisation

This execution time does not suit low-latency requirements, but that is how ACO is designed.

Session 1 - ML-based Systems

Service Placement Problem

Ant-Colony Optimisation

This execution time does not suit low-latency requirements, but that is how ACO is designed.

Session 1 - ML-based Systems

Service Placement Problem


We should analyse the problem first:

Session 1 - ML-based Systems

Service Placement Problem


We should analyse the problem first:

Variables we cannot reduce

  • Number of services
  • Number of iterations
  • Number of ants

Session 1 - ML-based Systems

Service Placement Problem


We should analyse the problem first:

Variables we cannot reduce

  • Number of services
  • Number of iterations
  • Number of ants

We can reduce the number of servers, how?

We can pre-select edge servers by predicting user locations.

Session 1 - ML-based Systems

Service Placement Problem


We should analyse the problem first:

Variables we cannot reduce

  • Number of services
  • Number of iterations
  • Number of ants

We can reduce the number of servers, how?

We can pre-select edge servers by predicting user locations.

ACO Smart City
Session 1 - ML-based Systems

Service Placement Problem

ACO Smart City
ACO Smart City
Session 1 - ML-based Systems

Service Placement Problem

MAACO Algorithm
Session 1 - ML-based Systems

Service Placement Problem

Selecting edge servers close to current and future users' location. We used two approaches that cluster historical trips and use these clusters to predict the next link in the user's path:

  • Bayesian Classifier
  • Hidden Markov Model
MAACO Algorithm
Session 1 - ML-based Systems

Service Placement Problem

MAACO results
Session 1 - ML-based Systems

Service Placement Problem


Bayesian Classifier


Hidden Markov Model

Session 1 - ML-based Systems

Service Placement Problem


Bayesian Classifier

  • Transition matrix depends on the number of streets in a city.

Hidden Markov Model

  • Frequency matrix depends on the number of streets in a city.
Session 1 - ML-based Systems

Service Placement Problem


Bayesian Classifier

  • Transition matrix depends on the number of streets in a city.
  • A lot of data (i.e., trips) are needed to train the model.
  • Training time is now an issue!
  • We assumed a limited number of streets in our work.

Hidden Markov Model

  • Frequency matrix depends on the number of streets in a city.
  • A lot of data (i.e., trips) are needed to train the model.
  • Training time is now an issue!
  • We assumed a limited number of streets in our work.
Session 1 - ML-based Systems

Service Placement Problem


Bayesian Classifier

  • Transition matrix depends on the number of streets in a city.
  • A lot of data (i.e., trips) are needed to train the model.
  • Training time is now an issue!
  • We assumed a limited number of streets in our work.

Hidden Markov Model

  • Frequency matrix depends on the number of streets in a city.
  • A lot of data (i.e., trips) are needed to train the model.
  • Training time is now an issue!
  • We assumed a limited number of streets in our work.

Again, new design decisions are needed to deploy these algorithms in the real-world.

Session 1 - ML-based Systems

Conclusions

Session 1 - ML-based Systems

Conclusions

Session 1 - ML-based Systems

Conclusions

Context People
Session 1 - ML-based Systems

Conclusions

AI Puzzle
Session 1 - ML-based Systems

Conclusions

Overview

  • ML Context
  • ML Today
  • ML Perception
  • ML Objective Definition
  • Context Matters
  • DSA Example
  • Sercice Placement Example
Session 1 - ML-based Systems

Conclusions

Overview

  • ML Context
  • ML Today
  • ML Perception
  • ML Objective Definition
  • Context Matters
  • DSA Example
  • Sercice Placement Example

Next Time

  • ML Adoption Process
  • Problem First
  • The Data Science Process
  • Data Orientation
  • Data Quality
  • A Machine Learning Pipeline
Session 1 - ML-based Systems

Many Thanks!

chc79@cam.ac.uk

_script: true

This script will only execute in HTML slides

_script: true