FOD#61: AI Fall: Time to Build

Next Week in Turing Post:

• Thursday, a guest post: Your infrastructure shouldn’t live in a black box

• Friday, AI Infra Unicorns: A Deep Dive into Graphcore

In partnership with

SciSpace is a next-gen AI platform for researchers where you can effortlessly browse 280 million+ papers, conduct effortless literature reviews, chat with, understand, and summarize PDFs with its AI copilot, and so much more. 

If you love it, get 40% off an annual subscription with code TP40 or 20% off a monthly subscription with code TP20.

NEW! Weekly recommendations from AI practitioner👍🏼:

• Cursor and Aider – they are similar (Aider is free, Cursor is paid - that’s what Copilot would have been) 

• Superwhisper – if you get it into your flow, then ah, that’s what Svoice assistant was supposed to be!

Twitter Library

Check our latest collections:

News from The Usual Suspects ©

• Google crushing it this week:

  1. Gemini 1.5 Pro outperforms competitors: GPT-4 and Claude 3.5 are behind in benchmarks.

  2. Gemma 2 2B – a smaller, safer, more transparent model. With ShieldGemma for content moderation and Gemma Scope for model interpretability, Google’s open-source push could redefine how we trust and understand AI. And it’s all wrapped in a neat, research-friendly package – talk about a gem!

  3. Google hires talent from Character.AI: Google's acquisition of top talent from Character.AI follows the trend. InflectionAI, AdeptAI, Stability (in a sense), CharacterAI – who’s next?

  4. Google Cloud expands database portfolio with new AI features, including graph and vector search in Spanner SQL.

  5. Google unveils three AI features for Chrome like Google Lens, natural language search in your history and Tab compare. Looks useful!

• GitHub Plays Hardball with AI

  • GitHub's new beta, GitHub Models, brings AI experimentation directly to developers’ fingertips. With Meta’s Llama 3.1 and OpenAI’s GPT-4o on tap, it’s a one-stop shop for AI model comparisons. By embedding AI tools seamlessly into its ecosystem, GitHub is aiming to outshine platforms like Hugging Face, making AI development as smooth as a single commit.

• Contamination Crisis in NLP

  • The 2024 CONDA Data Contamination Report uncovers a major issue: AI models like GPT-4 and PaLM-2 unknowingly feasting on evaluation data, leading to misleadingly high scores. With 91 sources contaminated, this report pushes for transparency and stricter evaluation methods in the NLP community. Consider it the AI world’s version of a doping scandal.

• Nvidia in the Hot Seat

  • Nvidia is in a bind, juggling antitrust probes and chip delays while secretly training robots with Apple’s Vision Pro. Their latest AI chip design flub might slow them down, but Nvidia’s influence in the tech world keeps growing – just not without some bumps along the way.

• Groq - congrats!

• From Stability to Flux

  • Former Stability.ai developers have founded Black Forest Labs, and announced the Flux.1 suite that is free and on par with Midjourney and DALL-E 3. The startup has secured $31 million in seed funding led by Andreessen Horowitz and plans to release text-to-video models next.

In other newsletters:

The freshest research papers, categorized for your convenience

Our top (all from Meta this week!)

• The Llama 3 Herd of Models

  This research paper introduces the "Llama 3 Herd," a collection of models based on the Llama 3 architecture. Meta's approach to making high-performance AI models accessible for fine-tuning by others represents a significant shift in the AI landscape. This move lowers the barrier to entry in the AI race, challenging the dominance of large tech companies by making it possible for smaller players to participate without the need for enormous financial resources.

• SAM 2: Segment Anything in Images and Videos

  Researchers from Meta FAIR introduced SAM 2, a unified model for segmenting objects in both images and videos. SAM 2 employs a transformer-based architecture with a streaming memory to efficiently process video frames, providing improved accuracy and reduced user interaction compared to previous models. The model is trained on the SA-V dataset, the largest video segmentation dataset to date. SAM 2 outperforms its predecessor, Segment Anything Model (SAM), in both speed and accuracy across various benchmarks.

• MoMa: Efficient Early-Fusion Pre-training with Mixture of Modality-Aware Experts

  Researchers from Meta FAIR introduced MoMa, a novel modality-aware mixture-of-experts (MoE) architecture for pre-training mixed-modal, early-fusion language models. MoMa utilizes modality-specific expert groups for text and image processing, improving efficiency with significant FLOPs savings compared to dense baselines. The model demonstrates superior pre-training efficiency and performance, particularly when combined with mixture-of-depths (MoD), though it faces challenges in causal inference. This approach paves the way for more resource-efficient multimodal AI systems.

• Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judge

  Researchers from Meta FAIR, UC Berkeley, and NYU propose "Meta-Rewarding," a method to enhance LLMs by allowing them to judge and refine their own judgments, improving their alignment and instruction-following abilities. The model's accuracy improved significantly on benchmarks like AlpacaEval 2 and Arena-Hard, achieving notable gains without relying on human supervision. This approach shows promise in developing self-improving models that can autonomously enhance their judgment and response quality.

A lot of research papers last week about non-english languages (including chemical language)

• JACOLBERTV2.5: Optimising Multi-Vector Retrievers to Create State-of-the-Art Japanese Retrievers with Constrained Resources improves Japanese language retrieval by optimizing multi-vector retrievers with constrained computational resources →read the paper

• A Large Encoder-Decoder Family of Foundation Models For Chemical Language introduces a family of transformer-based models for chemical tasks, achieving state-of-the-art performance in molecular property prediction and classification →read the paper

• SeaLLMs 3: Open Foundation and Chat Multilingual LLMs for Southeast Asian Languages introduces a multilingual model tailored for Southeast Asian languages, improving performance and efficiency in various tasks→read the paper

• ATHAR: A High-Quality and Diverse Dataset for Classical Arabic to English Translation provides a dataset for improving Classical Arabic to English translation, addressing gaps in existing resources →read the paper

• Sentiment Analysis of Lithuanian Online Reviews Using Large Language Models explores sentiment analysis for Lithuanian using fine-tuned transformer models, highlighting challenges in less-resourced languages →read the paper

• Meltemi: The first open Large Language Model for Greek develops Meltemi 7B, the first open-source LLM for Greek, showing strong performance on Greek language benchmarks →read the paper

• Adapting Safe-for-Work Classifier for Malaysian Language Text: Enhancing Alignment in LLM-Ops Framework creates a Safe-for-Work classifier for Malay text to detect unsafe content, improving safety in AI applications for Malaysia→read the paper

• Knesset-DictaBERT: A Hebrew Language Model for Parliamentary Proceedings fine-tunes a Hebrew language model on parliamentary texts for enhanced analysis of political discourse→read the paper

• Non Verbis, Sed Rebus: Large Language Models are Weak Solvers of Italian Rebuses assesses LLMs' performance on Italian rebuses, revealing limitations in sequential reasoning despite fine-tuning →read the paper

Language Model Improvements and Optimization

• Improving Text Embeddings for Smaller Language Models Using Contrastive Fine-tuning enhances text embeddings using contrastive fine-tuning for better performance in semantic similarity tasks →read the paper

• ThinK: Thinner Key Cache by Query-Driven Pruning optimizes memory usage in LLMs during inference by pruning less important cache channels based on query criteria →read the paper

Vision and Multimodal Models

• OmniParser for Pure Vision Based GUI Agent develops a vision-based method for parsing UI screenshots into structured elements, improving model performance across various applications →read the paper

• Mixture of Nested Experts: Adaptive Processing of Visual Tokens proposes a framework to enhance Vision Transformers by dynamically routing visual tokens to reduce computational costs while maintaining accuracy →read the paper

• Visual Riddles: A Commonsense and World Knowledge Challenge for Vision and Language Models rests vision-language models on a benchmark of riddles requiring commonsense and world knowledge, highlighting the challenge in integrating these aspects→read the paper

• Visual Riddles: A Commonsense and World Knowledge Challenge for Vision and Language Models introduces a benchmark challenging vision-language models with commonsense riddles, exposing gaps in model performance→read the paper