How Are AI Language Models Made?
Constructing a frontier language model is a multi-stage industrial process. No single team does it by hand as it requires petabytes of data (which is a lot to say the least), purpose-built compute clusters, and careful alignment work. Understanding the pipeline answers a cluster of common questions: how are AI language models trained, how are AI models created, how is AI programmed, and in which language are AI models written.
Stage 1: Pre-Training
Pre-training is where the model learns language itself. The training corpus typically contains web pages (Common Crawl), books (Books3, Gutenberg), code (GitHub), scientific papers (arXiv, PubMed), and curated high-quality text. Models are trained on trillions of tokens: Llama 3 70B was trained on 15 trillion tokens; GPT-4 reportedly on more than 13 trillion.
The training objective for autoregressive models is next-token prediction: given all previous tokens, predict the next one. The model computes a loss (cross-entropy) between its predicted distribution and the actual next token, then backpropagates gradients through its billions of parameters using optimisers like AdamW. This is done for millions of steps across the entire corpus.
Pre-training a frontier model costs tens to hundreds of millions of dollars in compute. GPT-4's training compute is estimated at roughly 2.15e25 floating-point operations.
Stage 2: Supervised Fine-Tuning (SFT)
A pre-trained model is a powerful but unruly text predictor. Fine-tuning on curated instruction-response pairs teaches it to be helpful, to follow instructions, and to behave appropriately in a conversational interface. SFT datasets are typically assembled by human annotators who write ideal responses to diverse prompts.
Stage 3: Alignment via RLHF
Reinforcement learning from human feedback (RLHF) trains a reward model that predicts which of two responses human raters prefer. The language model is then fine-tuned using proximal policy optimisation (PPO) to maximise this reward. For example, Anthropic's Constitutional AI (CAI) extends this with a set of written principles that when applied ultimately reduces reliance on people just having to go and manually label every decision. The result is a model that is helpful, honest, and avoids harmful outputs.
Which Programming Languages Are Used to Build LLMs?
Python is the dominant language for AI model development. Training and inference frameworks including PyTorch, JAX, and TensorFlow are all Python-first. The actual tensor operations, however, run as compiled CUDA kernels on NVIDIA GPUs (written in C++ and CUDA), so the true compute engine is lower-level than Python. R is used in academic research and statistical analysis but is not a mainstream training language. Rust is emerging as a language for inference runtimes that require performance and safety.
For those learning AI programming, Python with PyTorch is the clear entry point. Libraries like Hugging Face Transformers, LangChain, and the OpenAI and Anthropic SDKs make it possible to experiment with state-of-the-art models within hours of starting.
How to Create or Train Your Own AI Language Model
Training a frontier model from scratch is beyond the reach of most individuals and companies. However, there are several practical paths to building custom language systems. For example, fine-tuning an open-source base model using tools like Axolotl, LLaMA-Factory, or Hugging Face PEFT requires a dataset of instruction-response pairs and a GPU with 24GB or more VRAM for smaller models.
Retrieval-augmented generation (RAG) connects an existing model to a vector database of your documents, giving it access to private or specialized knowledge without retraining. Prompt engineering and system prompts customize model behavior within the constraints of an API, requiring no additional training at all. LoRA and QLoRA are parameter-efficient fine-tuning methods that reduce memory requirements dramatically by training only small adapter layers instead of the full model.
