- Turing Post
- Posts
- A Halfway Recap – Decoding FMOps: From Basics to Advanced RAG and CoT
A Halfway Recap – Decoding FMOps: From Basics to Advanced RAG and CoT
Organizing emerging knowledge in real-time
Ksenia Se & Valeriia Kuka
December 27, 2023
Four months ago, we launched our FMOps series to demystify the FMOps infrastructure stack. Our goal: to dissect its structure and assess its relevance and necessity. Writing about FMOps and what we term 'task-centered ML' is far from trivial, as it involves organizing emerging knowledge in real-time.
But it’s very insightful! We've covered a wide range of topics, from foundation model basics to advanced concepts like Retrieval-Augmented Generation (RAG) and Chain-of-Thought Prompting (CoT), extending to synthetic data. Fourteen editions are out, and more are in the pipeline!
It's time to recap and update new readers on our journey.
Before diving deeper, let's establish a glossary of key terms, akin to loading essential packages before starting main code development:
Foundation Models (FMs)
Large Language Models (LLMs)
Machine Learning (ML)
Foundation Model Operations (FMOps)
Large Language Model Operations (LLMOps)
Machine Learning Operations (MLOps)
Each Token has a free part with lots of insights, but to dig deeper, you need to be a premium subscriber. For two weeks only, we are offering a special 40% OFF → subscribe today for only $42 / YEAR (that’s only $3.5 / month)
In this opening edition, we explore the connection between emerging FMOps and LLMOps and established MLOps. We examine how FMOps creates a new ecosystem for foundation models, driven by task-centric ML. This shift promises to transform ML's economics, scalability, and applications.
This part examines the dynamic world of FMs, setting them apart from traditional ML models. We investigate scenarios where each excels, analyzing the reasons for their differing performances. The discussion also covers strategic considerations in choosing between traditional ML and FMs for various business scenarios, and highlights the transformative potential of FMs across multiple industries, backed by examples. Strategic insights.
Our first practice-focused edition examining Retrieval-Augmented Generation (RAG). We explore RAG's origins, its role in addressing LLMs’ limitations, its architectural design, and the reasons for its growing popularity. Additionally, you will find a curated collection of resources for RAG experimentation. Very popular!
A comprehensive guide for those new to foundation models. We cover systematic concepts such as the definition, key characteristics, and various types of FMs, enriched with practical insights from Rishi Bommasani, co-author of a seminal paper on foundation models, “On the Opportunities and Risks of Foundation Models.” Must-read for beginners.
This edition focuses on Chain-of-Thought Prompting (CoT), a concept proposed in 2022 that has recently gained popularity and transformed the field of prompting. We explore where CoT stands in the world of prompting starting from basic concepts like zero-shot and few-shot prompting to CoT’s various spin-offs. This edition serves as an up-to-date reference for understanding the complexities and developments in this evolving field. Everything you need to know about CoT.
We focus on Transformer and Diffusion-based models, leading players in the generative AI space. This edition explores the origins of their architectures, and how they function, and reviews some of the most significant implementations like Stable Diffusion, Imagen, and DALL-E. Core knowledge not to miss.
“This helped me gain a better understanding of what Tokenizer does in the encoding and decoding process. It was also amazing to see how the stable diffusion model works by adding noise.”
Following our discussion on Chain-of-Thought Prompting (CoT) in Token 1.5, we explore related concepts like Chain-of-Verification (CoVe) and Chain of Density (CoD). These concepts, though differing from CoT, use the 'chain' analogy to represent various methods of reasoning and summarization in LLMs.
“Have only seen CoT referenced previously. This made it much more concrete.”
A deep dive into the critical role of computing in FMs and LLMs. We examine the semiconductor industry's evolution, transitioning from general-use chips to specialized AI semiconductors. This edition covers market dynamics, computational choices, and what the future holds for AI chips. Answering the question: Why such GPU craze?!
This segment analyzes the critical decision between open- and closed-source FMs and its impact on AI strategies. We define the open- and closed-source models, outline key factors to consider when making this choice and explore the fundamental differences between open- and closed-source models for businesses. Where do you stand in the open- vs. closed-source battle?
Can small models substitute large models? We explain various techniques for downsizing models, including pruning, quantization, knowledge distillation, and low-rank factorization. Additionally, we provide insights on how to start with small models, bypassing the need to compress large ones and offer a list of smaller models suitable for various projects. Really useful!
In this edition, we compare various adaptation techniques for LLMs such as prompt engineering, fine-tuning, and retrieval-augmented generation (RAG). Through our analysis, we discovered that while prompt techniques and RAG are relatively easier to implement, fine-tuning stands out for its effectiveness in specific scenarios, despite its demanding requirements in terms of computational resources, memory, time, and expertise. We list and discuss the specific scenarios where fine-tuning is indispensable and then transition to how we can optimize this process with Low-Rank Adaptation (LoRA), a technique that has garnered significant interest recently. We will explore the origins of LoRA, explain how it operates, and illustrate why it's becoming an increasingly popular concept in the world of LLMs.
“The sequence of approaches is well written.”
Vector Databases are on fire. Why? Our discussion covers what vector databases are, their functionality, and their significance in managing complex data sets. We also look into alternative solutions and offer expert opinions on security aspects. For practical assistance, we provide a list of open-source vector databases and search libraries. Your best guide.
“Written with a steady hand!”
We tackle the crucial topic of data requirements for FMs, including the challenges posed by dataset biases and strategies for mitigating them. We discuss methods for data collection, introduce some data-efficient training techniques, and dive into the ethical aspects of data usage. Looking forward, we anticipate more in-depth discussions on data sourcing in the upcoming year. Data is the King!
We explore the concept of synthetic data, its origins, and its practical applications. This edition explains why synthetic data is needed, how to generate it, and the considerations for creating realistic synthetic datasets. Can you use only Synthetic data? Most likely no. Find out – why.
“Great overview of synthetic data. Highly recommend reading the full “article.
The FMOps series will continue in 2024! Happy Holidays, and best wishes for the New Year!
Send us your comments and suggestions about the topics you want to read on Turing Post 🤍
Reply