FOD#29: Microsoft Goes All In On AI, Now With Sam Altman and Greg Brockman as Employees

Remember when machine learning newsletters were actually about... machine learning?

Those were blessed times.

Now, AI has hit a corporate stage where it's all about a clash between the need for speed in product delivery and a more measured R&D pace, focusing on safety. And with a lot of power in play, the stakes are only getting higher. We're either on the verge of an AI winter or stepping into the era of actual AGI. But given that the bulk of recent ML news is more about human relationships, we could be witnessing the onset of corporate wars, complete with monopolistic grabs over this potent technology.

The big story, I believe, isn't just the OpenAI drama, though it's flipped the AI world on its head. It's about Microsoft's all-in bet on AI. And now, with Sam Altman's controversial exit from OpenAI, Microsoft seems to have brought both Sam Altman and Greg Brockman under its wing.

Satya Nadella has seemingly mastered the 'crouching tiger, hidden dragon' strategy, striking decisively at the most accurate time.

Microsoft’s Planned Moves Last Week

1. At Microsoft Ignite 2023, the company unveiled an expansive array of AI-driven developments, underscoring its deep investment in AI technology. As reported by TechCrunch, these announcements included:

   • Copilot AI Expansions: Microsoft introduced a series of new AI tools, dubbed 'Copilots,' designed to integrate AI into various business and consumer applications. This includes Copilot for Azure, a chatbot for cloud customers; Copilot for Service, aimed at enhancing customer service; Copilot in Dynamic 365 Guides for frontline workers; and Copilot Studio, a tool for creating custom chatbots.

   • Bing Chat's Renaming to Copilot: In a strategic move, Microsoft rebranded Bing Chat to Copilot, possibly signaling a shift in its competitive strategy against Google.

   • Custom Microsoft AI Chips: Introducing Maia 100 and Cobalt 100, Microsoft's in-house AI chips, marks a significant step in hardware development.

   • AI Upgrades for Teams: Microsoft Teams received AI-based enhancements, including a home decorator and noise reduction features.

   • Consolidation of Project Management Tools: Microsoft is merging To Do, Planner, and Project into a single, more streamlined product.

   • Windows AI Studio: This new tool allows for running AI models offline in Windows applications.

   • Azure AI Speech Text-To-Speech Avatars: This release, akin to a deepfake creator, represents a bold foray into generative AI.

2. Microsoft's current infrastructure expansion is nothing short of monumental. With a planned annual expenditure of over $50 billion for data centers starting in 2024, as reported by SemiAnalysis, this initiative far surpasses the investment levels of historical projects like national rail networks, massive dams, and even space endeavors like the Apollo moon missions. The primary aim of this massive buildout is to accelerate AGI's development and weave generative AI capabilities into all facets of life, from productivity tools to recreational activities.

Here Comes OpenAI Board Meltdown – Not Planned. Quick Recap

We are still in Monday. What’s happening next is unknown, especially considering Sam Altman’s recent tweets:

There a few main narratives right now:

• There is a problem with the board (e.g. reports from Semafor, Kara Swisher, Sharon Goldman)

• There is a problem with Sam Altman (e.g. coverage by The Newcomer) 

• Microsoft is making a ninja move! (e.g. analysis by CBInsights)

The Newcomer suggests we should slow down and assess the situation, especially with Sam Altman's departure from OpenAI. His exit raises concerns about the management of influential AI players. The emerging trust issues within OpenAI's board pose serious questions, particularly given OpenAI's ambitions in AGI. Stratechery underscores Microsoft's resources and intellectual property, positioning them as a potential AI leader. CB Insights bluntly describes Microsoft's strategy as an 'acqui-hire' of OpenAI, with Altman and Greg Brockman now closely tied to Microsoft. SemiAnalysis offers a bold assessment: OpenAI's internal disruption has led to Microsoft absorbing much of its research team.

Considering all the above, the past few days mark a significant consolidation of AI expertise under Microsoft, potentially reshaping the AI development landscape.

Additional Info: Since I expose a lot of Effective Altruism (EA) movements in the AI space, here is a marvelous article about Emmett Shear: New OpenAI CEO Was a Character in a Harry Potter Fanfic That's Wildly Popular With Effective Altruists

Somebody, stop them. It’s beyond ridiculous.

News from The Usual Suspects ©

What About Google? Delays

• Google's effort to launch Gemini, their answer to OpenAI's ChatGPT, faces delays and challenges, reports The Information. Initially promised to cloud customers and partners by November, its release has been pushed to early next year. This setback occurs as Google's cloud sales growth slows, contrasting with Microsoft's acceleration. Google's difficulties are compounded by the success of ChatGPT, which has become synonymous with AI among consumers and is used widely for various applications. Google's own AI product, Bard, has not achieved similar traction.

• Meanwhile, Google DeepMind introduced Lyria in a partnership with YouTube, as well as two new AI experiments, Dream Track and Music AI tools, aimed at enhancing creativity in music production. Lyria excels in generating high-quality music with intricate control over style and performance, while Dream Track allows creators to produce soundtracks using AI-generated voices and styles of various artists. The Music AI tools are designed to assist in the creative process, transforming simple inputs into complex musical outputs.

Additional Read: A stroll through Google's Model Garden by Artificial Ignorance

But Amazon is Restructuring Towards GenAI

• Amazon is restructuring its Alexa division, focusing more on generative AI and discontinuing certain initiatives, reports Geek Wire. This shift will result in the elimination of several hundred roles, as outlined in a memo by Daniel Rausch, Vice President of Alexa and Fire TV. The restructuring follows previous layoffs in Amazon's Devices & Services division, which faced significant financial losses. Amidst growing competition in generative AI, Amazon aims to enhance Alexa's capabilities, including a new feature for extended conversations.

Meta is Also Restructuring Towards GenAI

• Meta Platforms has restructured its AI teams, moving most employees from the Responsible AI team to its generative AI unit. This move, part of a broader reshuffle, raises concerns about how Meta will manage AI safety outside generative AI. The Responsible AI team, formed in 2019 to ensure fair and safe AI use, has faced challenges including layoffs, leadership changes, and a loss of high-profile advocates.

Twitter Library

Actual ML and AI news, categorized for your convenience

Reports and Surveys

1. Coatue's report: AI is The Coming Revolution. Here is three insights:

   • AI's Transformative Potential: AI is rapidly advancing, with applications like GitHub Copilot showing significant efficiency improvements. This progress suggests AI's potential to meaningfully impact various sectors, supported by increased executive interest in AI adoption.

   • Open Source in AI: The AI revolution is heavily reliant on open-source development, but there's a growing trend towards restricting access to AI resources. Despite this, the open-source AI community remains dynamic, playing a crucial role alongside proprietary models.

   • The Evolving AI Tech Stack: Understanding the AI-centric tech stack is key, with opportunities across all layers, from data centers to user applications. AI is moving towards 'intelligence-as-a-service,' requiring a focus on talent, data, and computing power. The rise of AI Ops tooling is democratizing AI, leading to a surge in AI application development and adoption.

2. Microsoft Research’s ‘The Impact of LLMs on Scientific Discovery: a Preliminary Study using GPT-4’: Analyzes GPT-4's potential in various scientific domains, highlighting its strengths and the need for fine-tuning for enhanced quantitative accuracy.

3. Shanghai Jiao Tong University and Ant Group’s Survey on Language Models for Code: Reviews over 50 models and their evolution in code processing, aiming to bridge NLP and software engineering communities.

Research in AI and Machine Learning

• EMU VIDEO: Presents a novel two-step process for text-to-video generation, producing high-quality videos efficiently →read more

• Video-LLaVA: Improves visual-language understanding by encoding images and videos into a unified feature space →read more

• The Chosen One: Offers a fully automated method for generating consistent characters from text prompts →read more

• Fast Chain-of-Thought: Enhances the efficiency of LLMs in reasoning tasks with a novel parallel and auto-regressive decoding method →read more

• SPHINX (MLLM): Blends model weights, tasks, and visual embeddings for enhanced vision-language alignment in multi-modal LLMs →read more

• Q-Instruct (Visual Abilities in LLMs): Focuses on improving low-level visual tasks in multi-modality foundation models like GPT-4V →read more

• Chain-of-Note: Presents a novel framework for improving the robustness of Retrieval-Augmented Language Models (RALMs) by generating sequential reading notes for retrieved documents →read more

3D Generation and Reconstruction

• MetaDreamer (Text-to-3D Generation): Efficiently creates high-quality 3D content from text prompts, addressing geometric and textural challenges →read more

• Drivable 3D Gaussian Avatars (D3GA): Introduces a 3D controllable model for creating realistic human avatars using Gaussian splats →read more

• DMV3D (3D Generation from Text/Images): Combines a transformer-based model with multi-view diffusion for fast and realistic 3D generation →read more

• Instant3D (Fast Text-to-3D Generation): A framework for rapidly generating 3D objects from text prompts using a feedforward network →read more

• One-2-3-45++ (Image-to-3D Textured Mesh): Transforms a single image into a high-quality 3D textured mesh rapidly →read more

Thank you for reading, please feel free to share with your friends and colleagues. In the next couple of weeks, we are announcing our referral program 🤍

Another week with fascinating innovations! We call this overview “Froth on the Daydream" - or simply, FOD. It’s a reference to the surrealistic and experimental novel by Boris Vian – after all, AI is experimental and feels quite surrealistic, and a lot of writing on this topic is just a froth on the daydream.