LLMs, RAGs, and AI Foundations

Mark module complete Expand all

10.1 AI, machine learning, deep learning and generative AI

10.2 Large language models, tokens and tokenisation

An LLM is a neural network trained on very large text (and often code/image) corpora to predict the next token given previous tokens. Through pre-training and later instruction tuning and human-feedback-based fine-tuning, it learns to follow instructions and generate useful text.

Tokenisation example (illustrative — exact splits depend on the tokenizer): “Unbelievably fast delivery!” might become ["Un", "believ", "ably", " fast", " delivery", "!"] → 6 tokens. A common rough rule for English is ~4 characters or ~¾ of a word per token, but it varies by tokenizer and language (many non-English scripts use more tokens per word — a cost and latency consideration for multilingual products).

10.2.1 How LLM inference works

10.2.2 Transformers and attention (conceptual)

The transformer architecture (Vaswani et al., 2017, “Attention Is All You Need”) uses self-attention: for each token, the model computes how much to “attend to” every other token in the context, letting it use relevant words regardless of distance. Stacking many attention layers lets the model build rich representations. Practical PM implications: context matters (instructions and examples in the prompt shape output); cost and latency grow with context length; and very long contexts can still lose details (“lost in the middle” effects have been reported), so retrieval and prompt structure still matter.

10.3 Embeddings, vector databases and semantic search

An embedding is a list of numbers (a vector) produced by a model to represent an item so that related items are close together. Similarity is commonly measured with cosine similarity:

• Vector database / index: stores embeddings and finds nearest neighbours quickly (approximate nearest-neighbour algorithms such as HNSW). Options: extensions to existing databases (e.g., pgvector for PostgreSQL), dedicated vector DBs, or search engines with vector support.
• Keyword search (BM25) is precise for exact terms (product codes, names); semantic search handles meaning and paraphrase; hybrid search combines both and often works best. A reranker can reorder the top candidates for relevance.

10.4 Prompt engineering

Techniques: clear instructions; few-shot examples; delimiters around retrieved text (and an instruction that text inside is data, not instructions); structured outputs (JSON); asking the model to think step by step or to plan before answering (useful for complex reasoning, at extra cost/latency); decomposition into smaller calls; self-check or second-pass review.

10.5 Retrieval-Augmented Generation (RAG)

RAG retrieves relevant information at question time and gives it to the model as context so the answer is grounded in specific, current, permitted sources.

Design decisions a PM shapes: which sources are approved (and who owns freshness); chunk size and overlap; metadata (date, product, region, access level); top-k; reranking; citation display; what happens when nothing relevant is found; escalation path; logging and privacy.

Common RAG failure modes: wrong chunk retrieved (vocabulary mismatch), right chunk missed (top-k too small), outdated or conflicting documents, chunk split mid-table, model ignores context, prompt injection hidden in a document, permission leaks.

10.6 Fine-tuning versus RAG versus prompting

Rule of thumb: start with prompting + a good evaluation set; add RAG for knowledge; consider fine-tuning when you have many high-quality examples, a stable task and evidence that it beats prompting on your eval.

10.7 Model evaluation and LLM evaluation methodologies

An evaluation (eval) is a repeatable set of test cases with scoring rules. A demo with three impressive answers is not evidence that an AI product works reliably.

Evaluation methods

10.8 Hallucinations and grounding

A hallucination is generated content that is false or unsupported by the provided sources. Grounding ties outputs to verifiable sources.

Mitigations: retrieve from approved sources; instruct to answer only from context and to say “I don't know”; require citations and verify them; constrain output formats; lower temperature for factual tasks; use tools for facts and maths (calculators, databases) rather than model memory; human review for high-stakes outputs; monitor and sample production answers.

10.9 AI safety, data privacy and security

• Safety: harmful or biased outputs, unsafe advice (medical, legal, financial), over-reliance, manipulation. Use content policies, refusals, safe completions, escalation.
• Prompt injection: malicious instructions inside user input or retrieved content (“ignore previous instructions…”). Mitigate with instruction hierarchy, delimiting untrusted text, least-privilege tools, output validation and human approval for actions (Module 12).
• Privacy: minimise personal data in prompts; check vendor data-use and retention terms; don't log sensitive data unnecessarily; honour deletion; data residency.
• Security: protect API keys; enforce access control outside the model.

10.10 Cost versus latency

10.11 AI product metrics

10.12 Human-in-the-loop (HITL) systems

HITL places people at key points: review before sending (drafts), approve actions (refunds above a threshold), handle escalations, label data for evaluation and improvement. Design the handoff: pass context so the customer doesn't repeat themselves; measure reviewer agreement and workload; use confidence thresholds to route.

10.13 AI product discovery, use-case prioritisation and model selection

Prioritising AI use cases — score on: user value; frequency/volume; data availability and quality; tolerance for error (cost of a wrong answer); feasibility (latency, integration); risk (safety, regulatory, reputational); measurability. High-value, high-tolerance, data-rich cases first (e.g., drafting replies for agents) before low-tolerance autonomous ones (e.g., credit decisions).

Model selection criteria: quality on your eval set; latency; cost; context window; modalities (text, image, audio); tool/function calling and structured output support; safety features; data-use and retention terms; deployment (API vs self-hosted/open-weight); regional availability; licence; vendor stability. Re-evaluate periodically — models change fast.

10.14 AI product lifecycle, production monitoring and governance

Production monitoring: sample and review answers weekly; track quality and safety metrics; watch for drift (new topics, changed policies, model version updates); alert on spikes in escalation or negative feedback; log prompts/responses with privacy controls; incident process for harmful outputs.

AI governance: documented intended use and limitations (model/system cards); risk assessment; human oversight; accountability (named owner); audit trails; bias and fairness testing; incident response; compliance with applicable law. Reference frameworks: the NIST AI Risk Management Framework (Govern, Map, Measure, Manage) and the EU AI Act, which takes a risk-based approach with obligations phased in over several years — seek qualified legal advice for specifics.

10.15 Exercises with model answers

10.16 Interview perspective

10.17 Key takeaways

• LLMs predict tokens; fluency ≠ truth. Ground, cite, evaluate.
• Embeddings enable semantic search; similarity ≠ correctness. Hybrid search often wins.
• RAG supplies knowledge at answer time and does not retrain models or require a vector DB; fine-tuning changes behaviour; start with prompting + evals.
• Evaluation is the backbone: golden sets, human rubrics, validated LLM judges, adversarial tests — weight failures by severity.
• Manage cost and latency in tokens; route, cache, trim context.
• Use HITL, permission-aware retrieval and governance; enforce access outside the model.

10.18 Check your understanding

10.19 Recommended resources

• Google: Machine Learning Glossary and Embeddings crash course (from your course references).
• Google Cloud: What is RAG? (course reference).
• Anthropic: Prompt engineering overview (course reference) and OpenAI: Prompt engineering guide.
• Vaswani et al., “Attention Is All You Need” (2017).
• NIST AI Risk Management Framework.
• OWASP Top 10 for LLM Applications.