Advanced

Covers model and engineering concepts

• Transformer

  A neural-network architecture that models relationships in sequences and underpins modern LLMs.

• Attention Mechanism

  A mechanism that lets a model focus dynamically on the most relevant parts of its input.

• Quantization

  A technique that represents model values with lower precision to reduce memory and inference cost.

• Linear Algebra

  A mathematical or data concept used to represent information, measure model error or improve training.

• Covariance

  A mathematical or data concept used to represent information, measure model error or improve training.

• Gradient

  A mathematical or data concept used to represent information, measure model error or improve training.

• Gradient Descent

  An optimisation method that repeatedly adjusts model parameters to reduce prediction error.

• Objective Function

  A mathematical or data concept used to represent information, measure model error or improve training.

• Learning Rate

  A mathematical or data concept used to represent information, measure model error or improve training.

• Regularization

  A mathematical or data concept used to represent information, measure model error or improve training.

• Bias-Variance Tradeoff

  A mathematical or data concept used to represent information, measure model error or improve training.

• Perceptron

  A building block or training mechanism used inside neural networks.

• Activation Function

  A building block or training mechanism used inside neural networks.

• ReLU

  A building block or training mechanism used inside neural networks.

• Sigmoid

  A building block or training mechanism used inside neural networks.

• Softmax

  A building block or training mechanism used inside neural networks.

• Backpropagation

  A building block or training mechanism used inside neural networks.

• Dropout

  A building block or training mechanism used inside neural networks.

• Batch Normalization

  A building block or training mechanism used inside neural networks.

• Convolutional Neural Network

  A building block or training mechanism used inside neural networks.

• Recurrent Neural Network

  A building block or training mechanism used inside neural networks.

• Long Short-Term Memory

  A building block or training mechanism used inside neural networks.

• Gated Recurrent Unit

  A building block or training mechanism used inside neural networks.

• Encoder-Decoder

  A building block or training mechanism used inside neural networks.

• Byte Pair Encoding

  A concept used to represent, process or generate human language with machine-learning models.

• WordPiece

  A concept used to represent, process or generate human language with machine-learning models.

• SentencePiece

  A concept used to represent, process or generate human language with machine-learning models.

• Positional Encoding

  A concept used to represent, process or generate human language with machine-learning models.

• Self-Attention

  A concept used to represent, process or generate human language with machine-learning models.

• Multi-Head Attention

  A concept used to represent, process or generate human language with machine-learning models.

• Masked Language Model

  A concept used to represent, process or generate human language with machine-learning models.

• Causal Language Model

  A concept used to represent, process or generate human language with machine-learning models.

• Mixture of Experts

  A model architecture or technique that shapes how an AI system represents, generates or retrieves information.

• Sparse Model

  A model architecture or technique that shapes how an AI system represents, generates or retrieves information.

• Generative Adversarial Network

  A model architecture or technique that shapes how an AI system represents, generates or retrieves information.

• Variational Autoencoder

  A model architecture or technique that shapes how an AI system represents, generates or retrieves information.

• Autoencoder

  A model architecture or technique that shapes how an AI system represents, generates or retrieves information.

• Vision Transformer

  A model architecture or technique that shapes how an AI system represents, generates or retrieves information.

• Graph Neural Network

  A model architecture or technique that shapes how an AI system represents, generates or retrieves information.

• Retrieval Model

  A model architecture or technique that shapes how an AI system represents, generates or retrieves information.

• Reranker

  A model architecture or technique that shapes how an AI system represents, generates or retrieves information.

• Cross-Encoder

  A model architecture or technique that shapes how an AI system represents, generates or retrieves information.

• Bi-Encoder

  A model architecture or technique that shapes how an AI system represents, generates or retrieves information.

• Knowledge Distillation

  A model architecture or technique that shapes how an AI system represents, generates or retrieves information.

• Pruning

  A model architecture or technique that shapes how an AI system represents, generates or retrieves information.

• Model Merging

  A model architecture or technique that shapes how an AI system represents, generates or retrieves information.

• Ensemble Learning

  A model architecture or technique that shapes how an AI system represents, generates or retrieves information.

• Reinforcement Learning from Human Feedback

  A method or artifact used to train a model and adapt its behavior to a task.

• Direct Preference Optimization

  A method or artifact used to train a model and adapt its behavior to a task.

• Low-Rank Adaptation

  A method or artifact used to train a model and adapt its behavior to a task.

• Parameter-Efficient Fine-tuning

  A method or artifact used to train a model and adapt its behavior to a task.

• Domain Adaptation

  A method or artifact used to train a model and adapt its behavior to a task.

• Curriculum Learning

  A method or artifact used to train a model and adapt its behavior to a task.

• Hyperparameter Tuning

  A method or artifact used to train a model and adapt its behavior to a task.

• Learning Rate Schedule

  A method or artifact used to train a model and adapt its behavior to a task.

• Greedy Decoding

  A setting or system component that affects how a trained model produces and serves outputs.

• Beam Search

  A setting or system component that affects how a trained model produces and serves outputs.

• KV Cache

  A setting or system component that affects how a trained model produces and serves outputs.

• Continuous Batching

  A setting or system component that affects how a trained model produces and serves outputs.

• Reciprocal Rank Fusion

  A component or method used to find, rank and ground information for an AI answer.

• On-device Inference

  A setting or system component that affects how a trained model produces and serves outputs.

• Instruction Hierarchy

  A concept for giving instructions, context, structure or safety boundaries to a generative model.

• Hybrid Search

  A component or method used to find, rank and ground information for an AI answer.

• Cosine Similarity

  A component or method used to find, rank and ground information for an AI answer.

• Approximate Nearest Neighbor

  A component or method used to find, rank and ground information for an AI answer.

• Vector Index

  A component or method used to find, rank and ground information for an AI answer.

• Ranker

  A component or method used to find, rank and ground information for an AI answer.

• Autonomous Agent

  A component or pattern for letting an AI system plan, use tools and complete multi-step work.

• Multi-agent System

  A component or pattern for letting an AI system plan, use tools and complete multi-step work.

• Agent Evaluation

  A component or pattern for letting an AI system plan, use tools and complete multi-step work.

• Area Under the Curve

  A concept for measuring quality, reliability, risk or responsibility in an AI system.

• Perplexity

  A concept for measuring quality, reliability, risk or responsibility in an AI system.

• BLEU

  A concept for measuring quality, reliability, risk or responsibility in an AI system.

• ROUGE

  A concept for measuring quality, reliability, risk or responsibility in an AI system.

• Adversarial Attack

  A concept for measuring quality, reliability, risk or responsibility in an AI system.

• Latent Diffusion

  A method or capability for working with images, audio, video or multiple data types.

• Data Drift

  An engineering practice or platform component for building, deploying and operating AI systems.

• Concept Drift

  An engineering practice or platform component for building, deploying and operating AI systems.

• Feature Store

  An engineering practice or platform component for building, deploying and operating AI systems.

• Kubernetes

  An engineering practice or platform component for building, deploying and operating AI systems.

• Distributed Training

  An engineering practice or platform component for building, deploying and operating AI systems.

• Federated Learning

  An engineering practice or platform component for building, deploying and operating AI systems.