FOD#64: Golden Age for Indie Devs and Engineers

This Week in Turing Post:

• Wednesday, AI 101: a deep dive into the DeepSeek family

The main topic

The rise of tools like Cursor AI and easy-to-make homemade robots is fueling a renaissance for indie developers and engineers. We tipped you off about Cursor almost a month ago, but only now has this indie hacker tool catapulted into the spotlight, igniting buzz across Twitter. From 8-year-olds to AI experts like Karpathy, everyone seems smitten. For many, there's no turning back to the old ways of coding, though, of course, there are dissenters in the mix.

From an indie hacking tool, Cursor AI has fancied up to become an a16z-backed startup. It reached a $400 million valuation after a $60 million Series A round with additional backing from Thrive Capital, the OpenAI Startup Fund, and Google’s Chief Scientist Jeff Dean. Cursor AI is transforming developers into coding virtuosos, with real-time code generation, on-the-fly debugging, and a lot fewer headaches. It’s a game-changer for software development, making coding more intuitive and efficient by automating repetitive tasks and understanding developer intent. What if you don’t have any coding experience? Cursor AI is good for you too.

But that's not all. A new force is democratizing robotics: Hugging Face's LeRobot (remember to say it with a French accent for full effect). This cutting-edge toolkit brings AI-powered robotics to your fingertips, offering pre-trained models, human-collected datasets, and simulation environments. Whether you're a developer, educator, or hobbyist, LeRobot makes building and teaching robots easier than ever, lowering the entry barrier and making robotics more accessible and practical.

For those who once built their own radio station or cobbled together a Frankenstein of a computer in a dorm room, the future is here: you can now build a robot in your apartment. Check their GitHub here.

Oh, if you can get good at that fast, keep in mind that over 70 legit robotics companies are hiring right now. What is that if not a signal of a boom in the industry?

Now, what does this mean for you, the fearless indie developer? You're no longer stuck in the shadows, battling against corporate giants. With LeRobot and Cursor AI in your toolkit (and other tools like this), you're equipped to compete on a whole new level.

Imagine this: you, fueled by ramen and raw ambition, crafting the next killer app in your pajamas. Your robot assistant, powered by LeRobot, fetches you another coffee while Cursor AI effortlessly cranks out flawless code. LeRobot might still move a bit too slow while folding a shirt but every day brings new achievements, and pretty soon it might become really helpful.

We are continuously updating this thread on Twitter with good examples of using Cursor, and we encourage you to go and create some digital magic—the world is ready and waiting. Share your results with us, and we'll give you a shoutout.

Our Twitter library

Weekly recommendation from AI practitioner👍🏼:

• n8n.io is an open-source workflow automation tool that democratizes the creation of AI-driven workflows, enabling users – even non-tech – to build intelligent agents and automate tasks across various apps and services, much like an AI-powered version of managing a business through a simple spreadsheet.

If any of this is helpful, please forward this email to a colleague. That allows us to keep Monday’s digest free for everyone.

News from The Usual Suspects ©

• Grok Accelerates with SGLang

  Grok 2 mini has found its groove, doubling its speed thanks to a rewrite of its inference stack using SGLang. Developed in record time by the duo of @lm_zheng and @MalekiSaeed, this upgrade not only boosts performance but also opens the door to serving the larger Grok 2 model with ease. Swift moves for a model with swagger.

• Andreessen Horowitz's Gen AI Spotlight: New Kids on the Block

  The latest Top 100 Gen AI Consumer Apps list from Andreessen Horowitz highlights a wave of innovation, with nearly 30% of companies being newcomers. Creative tools dominate, but the real buzz is around AI assistants, as Perplexity and Claude surge in popularity, challenging the reigning ChatGPT.

• Aleph Alpha Goes Multilingual with Pharia Models

  Aleph Alpha introduces Pharia-1-LLM-7B-control, designed for concise, controlled responses in European languages. With open licensing, these models aim for transparency and ethical use, aligning with forthcoming EU AI regulations. Aleph Alpha’s move positions it as a key player in ethical AI, catering to industries that value compliance and clarity.

• Google AI's New Prompt Gallery: A Creative Showcase

  Google's AI Studio offers a buffet of pre-built prompts for the Gemini API. It's a smorgasbord of creativity, perfect for developers looking to jumpstart their projects with Google's latest AI innovations. Gemini promises to be the wingman your next big idea needs.

• California AI Bill: A Clash of Titans

  *tense music playing

  California’s SB 1047, a bill to regulate AI, has tech heavyweights divided. Anthropic and AI godfather Yoshua Bengio (an interview) back the bill, while Meta and AI godmother Fei-Fei Li (an opinion) voice concerns.

• OpenAI Fine-Tuning: Custom-Made Intelligence

  OpenAI now offers fine-tuning for GPT-4o, letting developers mold the AI to fit specific needs. With perks like enhanced accuracy and domain-specific finesse, it’s an open invitation to make AI work harder for you. For now, training tokens are on the house.

• Andrew Ng’s New Chapter: From CEO to Executive Chairman

  AI luminary Andrew Ng steps down as CEO of Landing AI, transitioning to Executive Chairman. With COO Dan Maloney taking the reins, Ng’s pivot suggests a deeper focus on his AI Fund and Visual AI research. Industry watchers are on high alert – Ng’s next moves could signal the dawn of a new AI venture.

We are watching/reading:

• FermiNet: Quantum physics and chemistry from first principles by Google DeepMind

• What’s Really Going On in Machine Learning? Some Minimal Models by Stephen Wolfram

• ‘Make your health insurance company cry’: One woman’s fight to turn the tables on insurers about our reader Holden Karau

The freshest research papers, categorized for your convenience

Language Models and Their Innovations

• Introducing Pharia-1-LLM: transparent and compliant 

  Aleph Alpha launches Pharia-1-LLM-7B-control and Pharia-1-LLM-7B-control-aligned models, emphasizing multi-lingual capabilities and ethical AI compliance in line with EU regulations. Read the paper.

• Discover the New Multi-Lingual, High-Quality Phi-3.5 SLMs
  Introduces Microsoft's Phi-3.5 series, including models optimized for multi-lingual tasks, image understanding, and high-performance across various domains using a Mixture-of-Experts approach. Read the paper.

• HERMES 3 Technical Report
  Describes the Hermes 3 model with advanced reasoning, creativity, and alignment capabilities, excelling in benchmarks and leveraging up to 405B parameters for various applications. Read the paper.

Our top

• Automating Thought of Search: A Journey Towards Soundness and Completeness introduces AutoToS, an extension that automates the creation of search components, ensuring accuracy and completeness without human feedback. Read the paper.

• To Code, or Not To Code? Exploring Impact of Code in Pre-training explores how incorporating code in LLM pre-training significantly boosts performance across coding and non-coding tasks, highlighting the benefits of code in model generalization. Read the paper.

• Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion, and Aviation introduces CrossFormer, a transformer model capable of controlling diverse robotic platforms, demonstrating adaptability and performance across various real-world tasks. Read the paper.

Enhancing AI Model Capabilities

• D5RL: Diverse Datasets for Data-Driven Deep Reinforcement Learning proposes the D5RL benchmark to evaluate offline deep reinforcement learning using diverse, realistic datasets for robotic tasks, focusing on task variability and policy robustness. Read the paper.

• Jamba-1.5: Hybrid Transformer-Mamba Models at Scale introduces Jamba-1.5 models with a hybrid architecture and a novel quantization technique, achieving high performance in long-context and standard benchmarks. Read the paper.

Advanced Image Processing and Multimodal Models

• Photorealistic Object Insertion with Diffusion-Guided Inverse Rendering presents DiPIR, a method for realistic object insertion into images using large diffusion models to guide inverse rendering, enhancing applications in virtual production. Read the paper.

• Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model introduces Transfusion, a multi-modal model combining language modeling and diffusion techniques, achieving high-quality text and image generation. Read the paper.

Strategies for Robustness and Efficiency in AI Models

• MagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative Decoding proposes a method using speculative decoding to improve latency and throughput in long-context LLMs, demonstrating significant speedup without compromising accuracy. Read the paper.

• Enhancing Robustness in LLMs: Prompting for Mitigating the Impact of Irrelevant Information develops the GSMIR dataset and ATF prompting method to enhance LLMs' ability to filter out irrelevant information, improving reasoning accuracy. Read the paper.

Novel AI Applications and Techniques

• STRATEGIST: Learning Strategic Skills by LLMs via Bi-Level Tree Search introduces a method for developing strategic skills in LLMs using self-play and bi-level tree search, outperforming traditional reinforcement learning. Read the paper.

• SHORTCIRCUIT: AlphaZero-Driven Circuit Design presents a model for efficient Boolean circuit design using AlphaZero techniques, reducing circuit size significantly compared to state-of-the-art methods. Read the paper.

Innovations in Model Architecture and Optimization

• FocusLLM: Scaling LLM’s Context by Parallel Decoding introduces a framework extending context length using parallel decoding, handling sequences up to 400K tokens efficiently with improved accuracy. Read the paper.

• The Vizier Gaussian Process Bandit Algorithm enhances Google Vizier's Bayesian optimization with a scalable Gaussian Process Bandit Algorithm, optimizing complex, high-dimensional tasks. Read the paper.