FOD#47: AI Goes Mainstream: Celebrity Models, Beer-Brewing Algorithms, and the Future We Live In

Next Week in Turing Post:

• Wednesday, Recap#2: FMOps Infrastructure (visualized)

• Friday: An interview with Jensen Huang, Sam Altman, and Satya Nadella

Where has it been seen that the launch of an open-source model would be covered in the press in the style of a reportage? But here we are: WIRED covering the launch of DBRX, a new open-sourced model from Databricks.

This level of transparency and public relations is something new in the AI world. Is it a clever marketing move, or are algorithms truly becoming the hottest stars in town? Jensen Huang, at the opening of GTC, ‘reminded’ attendees, "I hope you realize it's not a concert but a developer conference. There will be a lot of science, algorithms, computer architecture, mathematics." A prepared joke that highlights that everything about GTC and Jensen Huang himself was set as a rock concert. In this light, it’s not surprising that Demis Hassabis, Google DeepMind CEO, has just been knighted. And that every US federal agency must now hire a chief AI officer. Each day, AI keeps making headlines. But considering the amount of science stuffed into this pie, it's a type of celebrity I can totally become a fan of.

Not that long ago, Yann LeCun, Meta’s Chief AI Scientist, found his 100-meter portrait displayed on the Burj Khalifa during the World Government Summit in Dubai. When I have a chance to interview him, I’ll ask if he could have imagined, in 1989, demonstrating the practical application of backpropagation at Bell Labs, that he – a nerd – would become such a star.

In fantastic times, we live. With AI permeating every aspect of our lives.

But leave all this Dubai exaggeration aside: you know AI is really getting serious when Belgian brewmasters leverage it to enhance beer flavors. They just used ML to analyze 250 Belgian beers for chemical composition and flavor attributes, to predict taste profiles and appreciation. Cheers to that!

Today, we won't offer you any architecture explanations, instead we encourage you to dedicate some time to these videos, which offer a glimpse into the future we're already living in. I plan to watch these with my kids. If they switch from wanting to be YouTubers to becoming 'hot' AI scientists, I'll be fully supportive.

• For an evening with family:

  • 2 hours of a blockbuster with Jensen Huang and his Nvidia GTC Keynote

  • A school project: A little guide to building Large Language Models in 2024 by Tom Wolf from Hugging Face

• For an evening with friends (who are also interested in investing): AI Ascent 2024 by Sequoia, including:

  • The AI opportunity: Sequoia Capital's AI Ascent 2024 opening remarks

  • What's next for AI agentic workflows - Andrew Ng, AI Fund

  • Trust, reliability, and safety in AI - Daniela Amodei, Anthropic and Sonya Huang

  • Making AI accessible - Andrej Karpathy and Stephanie Zhan

  • Open sourcing the AI ecosystem - Arthur Mensch, Mistral AI and Matt Miller

  • What's next for photonics-powered data centers and AI - Nick Harris, Lightmatter

  • What's next for AI agents - Harrison Chase, LangChain

• For a car ride: Sam Altman: OpenAI, GPT-5, Sora, Board Saga, Elon Musk, Ilya, Power & AGI | Lex Fridman Podcast #419 

• If you still have three hours left: Sholto Douglas (DeepMind) & Trenton Bricken (Anthropic) – How to Build & Understand GPT-7's Mind

And if you want someone grumpy about AI, you can attend to these two posts by Gary Marcus (about GenAI bubble and the race between positive and negative), who also thinks he is an AI celebrity.

Twitter Library

Hottest Releases of the Week (pam-pam-pam!):

Databricks with Mosaic’s DBRX

DBRX is a state-of-the-art open LLM by Databricks, outperforming GPT-3.5 and rivaling Gemini 1.0 Pro, especially in coding tasks. Its fine-grained Mixture-of-experts (MoE) architecture enhances efficiency, offering 2x faster inference than LLaMA2-70B with significant size reduction. DBRX excels across various benchmarks due to its training on a curated 12T token dataset. It's available on Hugging Face, integrating seamlessly into Databricks' GenAI products, marking a leap in open-source LLM development.

The former CEO of Mosaic, now a Databricks VP, commented on the outstandingly low $10 million spent on training DBRX:

→Read one of our most famous profiles: Databricks: the Future of Generative AI in the Enterprise Arena

JAMBA news

Just last week, we discussed the mamba architecture that rivals the famous transformer. This week, the news is even more impressive: AI21 introduced a mix of the two: Jamba, AI21's pioneering SSM-Transformer model, merges Mamba SSM technology with the Transformer architecture, offering a substantial 256K context window. It outperforms or matches leading models in efficiency and throughput, achieving 3x throughput on long contexts. Unique for fitting 140K context on a single 80GB GPU, Jamba democratizes AI with its open weights and hybrid architecture. Here you can read the paper.

Other impressive releases (both on March 28):

xAI released Grok-1.5

(though there are discussions about how trustworthy current benchmarks are)

Qwen released Qwen1.5-MoE

Speaking about Chinese LLMs:

News from The Usual Suspects ©

Microsoft's New Azure AI Tools

• Announced tools enhance generative AI app security: Prompt Shields for injection attacks, Groundedness detection, Safety templates, Evaluations for risks, and Monitoring. Aims to secure AI goals against risks.

OpenAI's Voice Engine

• Introduces a model for generating natural speech from text and audio samples, cautiously previewing to prevent misuse. Targets diverse applications, ensuring safety with consent and watermarking for traceability.

Chips – "DeepEyes"  

• Chinese company Intellifusion launches a cost-effective AI processor, 90% cheaper than GPUs, sidestepping U.S. sanctions. Aims for wide AI market impact, highlighting China's push for affordable, domestic AI technology.

The freshest research papers, categorized for your convenience

Our top-3

BioMedLM: A 2.7B Parameter Language Model Trained On Biomedical Text

Researchers from Stanford University and DataBricks introduced BioMedLM, a 2.7 billion parameter GPT-style language model specifically trained on biomedical texts from PubMed abstracts and articles. Unlike larger, general-purpose models like GPT-4 or Med-PaLM 2, BioMedLM offers a targeted, efficient, and privacy-preserving solution for biomedical NLP tasks, achieving competitive results on multiple-choice biomedical question-answering benchmarks. For instance, it scores 57.3% on MedMCQA (dev) and 69.0% on the MMLU Medical Genetics exam. The model's specialized training enables it to effectively answer patient queries on medical topics and represents a significant step towards smaller, domain-specific models that are both high-performing and resource-efficient →read the paper

AutoBNN: Probabilistic Time Series Forecasting with Compositional Bayesian Neural Networks

AutoBNN, developed by Google Research, represents a significant step forward in time series forecasting by merging the interpretability of Gaussian Processes with the scalability of neural networks. This framework could revolutionize how we approach forecasting problems by offering a more accurate and interpretable method, especially valuable for applications requiring rigorous uncertainty estimation, such as financial markets or weather forecasting →read the blog

Learning from interaction with Microsoft Copilot (web)

The work on Microsoft Copilot showcases a pioneering exploration into improving AI through user interaction, highlighting the shift towards more dynamic, responsive, and user-informed AI systems. This research could redefine user interfaces, making AI systems not just tools but collaborators in knowledge work and beyond, indicating a new direction in human-AI interaction →read the blog

Large Language Model (LLM) Innovations

• Gecko: Versatile Text Embeddings Distilled from Large Language Models: A compact model from Google DeepMind that efficiently distills LLM knowledge for improved information retrieval. read the paper

• Updating Large Language Models by Directly Editing Network Layers: Introduces SaLEM, a method for quick, efficient LLM updates by editing salient layers. read the paper

• AIOS: LLM Agent Operating System: Aims to optimize LLM agent deployment and integration for enhanced performance by Rutgers University. read the paper

• Long-form Factualty in Large Language Models: Google DeepMind and Stanford University's approach to reducing factual errors in LLM responses. read the paper

• sDPO: Don’t Use Your Data All at Once: Presents a novel approach for aligning LLMs with human preferences using a stepwise data utilization method. read the paper

• LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement: UC Berkeley's strategy for enhancing LLM performance through iterative data augmentation. read the paper

• The Unreasonable Ineffectiveness of the Deeper Layers: An empirical study on the minimal impact of removing LLM layers on performance. read the paper

• Can Large Language Models Explore In-Context?: Investigates LLMs' capability for exploration in reinforcement learning scenarios. read the paper

Multimodal Models and Information Retrieval

• Are We on the Right Way for Evaluating Large Vision-Language Models?: Critiques current LVLM benchmarks and introduces MMStar for a more comprehensive evaluation. read the paper

• Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models: Enhances VLMs for improved performance in multi-modal tasks by The Chinese University of Hong Kong and SmartMore. read the paper

• FOLLOWIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions: A dataset and framework to improve IR models' adherence to complex instructions. read the paper

• AllHands: Ask Me Anything on Large-scale Verbatim Feedback via Large Language Models: Microsoft's framework for analyzing large-scale user feedback using an LLM interface. read the paper

• LITA: Language Instructed Temporal-Localization Assistant: NVIDIA's approach to improving temporal localization in video content using LLMs. read the paper

Performance Optimization and Real-World Applications

• Fully-fused Multi-Layer Perceptrons on Intel Data Center GPUs: Demonstrates significant performance optimization of MLPs on Intel GPUs. read the paper

• A comparison of Human, GPT-3.5, and GPT-4 Performance in a University-Level Coding Course: Evaluates GPT variants against human performance in coding assignments. read the paper

• Towards a World-English Language Model for On-Device Virtual Assistants: Develops a unified language model for various English dialects for virtual assistants by AppTek GmbH and Apple. read the paper