Democratizing AI: The Hugging Face Ethos of Accessible Machine Learning

This is our fourth Corporate Chronicle (#1 OpenAI, #2 Anthropic, #3 Inflection) focusing on generative AI unicorns. We initially planned to spotlight Cohere, but Hugging Face disrupted the narrative. Because of that, the publication is hitting your inbox on Saturday instead of Friday, our apologies.

In August 2023, Hugging Face secured $235 million in a Series D round, boasting a valuation of $4.5 billion. It surged ahead, not just of Cohere but surpassing Inflection, with its ~$4.0 billion valuation.

In emoji terms, all we can say is this 👏 

According to our own Twitter sentiment analysis, Hugging Face is a widely admired player in both ML and AI. Everybody seems to love them.

• How did they achieve it?

• What makes this light-looking company such a heavyweight?

• Who are the people behind it, and what is their vision? What did they build?

• Are they responsible for the AI they work on? Are they reliable?

• How do they plan to make money? etc

We'll answer these questions and more in our exploration of Hugging Face. Tune in.

1. The starting point of the company

2. Mission of the company and its founders

3. Groundbreaking Initiatives

4. The tech behind their most famous products

5. Current situation including finances

6. How the company is going to make money?

7. Founders’ attitude toward AI risks

8. Regulations

9. Bonus: All important links about the founders

The starting point of the company

“I’m French, as you can hear from my accent, and moved to the US 10 years ago, barely speaking English,” said Clement Delangue, co-founder and CEO at Hugging Face, in his testimony to the US House of Representatives in June 2023. He was working for a startup called ‘Mention’ then, in 2013, that was acquired by ‘Mynewsdesk’ in 2014. He was CMO with a passion for sharing knowledge and building communities with maximum openness (Check this spotlight story from Sequoia Capital if you want to learn about Delangue’s childhood.)

In 2015, he and his long-term friend Julien Chaumond decided to enroll in a computer science class created by Richard Socher. Recently, Richard tweeted:

In 2015, though, they were building a chatbot designed to be a fun companion for teenagers. If you remember 2015, Conversational AI was not yet well-developed. Their attempt was gutsy. The three founders – Thomas Wolf, who knew Julien from École Polytechnique and was also part of the Stanford learning group – dug deep into natural language processing (NLP) to bring their whimsically-faced chatbot to life. In 2016, Hugging Face was launched. With scientific credibility backed by Richard Socher, they raised $1.2 million in 2017 from SV Angel, Betaworks, and NBA star Kevin. "We really have this vision where we believe everyone will have an AI friend and interact daily with Hugging Face," Delangue said. If only he knew that with almost the same objective, a company called Inflection would raise over $1.5 billion in 2023... But in 2016-2017, to power their bot, three founders had to explore every inch of the NLP universe.

Making their own research public was just natural for them. “Sharing knowledge benefits everybody” was part of their attitude towards business.

2017 changed everything in NLP – a transformative year punctuated by the unveiling of 'transformers,' a novel architecture conceived by researchers from Google and the University of Toronto and explained in their infamous paper “Attention is All You Need.” Transformers provided the scaffolding for formidable language models like BERT, ROBERTa, and GPT-3, all of which had the innate ability to understand the intricacies of language at a blistering speed, courtesy of parallel GPU processing.

These models were monumental but also monumental-ly inaccessible to the average Joe Developer. Working on their chatbot, Hugging Face developed an open-source transformers library, putting the power of cutting-edge NLP into the hands of developers regardless of their Silicon Valley pedigree. The game-changer, however, came when Hugging Face simplified Google's BERT model, refactoring it into PyTorch, and—gasp—open-sourced it.

Mission of the company and its founders

From this moment their mission crystallized from being mere chatbot developers to evangelists of accessible machine learning. "It just seemed to be something a lot of people would like to use," Delangue remarked, his words betraying a nonchalant profundity. And so, the mission was redefined: Hugging Face would be a sanctuary for machine learning enthusiasts, whether engineers, researchers, or tinkerers at a weekend hackathon. Since then they aimed to become the GitHub of machine learning.

This pivot ushered in an era of symbiotic cross-pollination within the community. Now Hugging Face wasn't just a tool; it was a movement, a pulsating hub of ingenuity. And just like that, a company transformed itself by transforming the very way we access and leverage machine learning. It's a fascinating narrative of how a change in mission can make not just a company, but an entire community, punch above its weight.

And when they now say it’s the fastest-growing community and most used platform for machine learning – it’s actually true.

In an interview in March, 2021 Hugging Face CTO Julien Chaumond said that the democratization of AI will be one of the biggest achievements for society. He added that no single company, not even a Big Tech business, can do it alone.

Their official mission from the website states:

Groundbreaking Initiatives

April 28, 2021 – the company launched the BigScience Research Workshop in collaboration with several other research groups to work on an open large language model.

In 2022, the workshop concluded with the announcement of BLOOM, a multilingual large language model with 176 billion parameters.

On May 13, 2022, the company introduced its Student Ambassador Program to help fulfill its mission to teach machine learning to 5 million people by 2023.

They also have an impressive list of partnerships, including StabilityAI, AWS, and others that are tremendously beneficial for their community.

The tech behind their most famous products

About BLOOM: When we imagine the origins of large language models (LLMs), we often picture Silicon Valley giants or flush startups. But BLOOM shatters this narrative. Spawned by BigScience, a consortium of 1,200 researchers from 38 countries, this open-source model boasts 176 billion parameters and speaks 46 languages plus 13 coding tongues.

Under the hood, BLOOM is more than just a heavyweight. It employs causal-decoder transformer models but adds two innovations: ALiBi Positional Embeddings and Embedding LayerNorm, elevating both training and performance. The model trained on the ROOTS corpus, a smorgasbord of 498 datasets, adding linguistic diversity to its portfolio.

Behind the scenes, the Megatron-DeepSpeed framework orchestrates a symphony of efficient large-scale training, with a Zero Redundancy Optimizer as the conductor.

Megatron-LM provides the core Transformer implementation. It also enables tensor parallelism, allowing the model to split individual layers across multiple GPUs. This horizontal slicing of layers enables a larger-than-memory model to train efficiently.

Types of Parallelism:

• Data Parallelism (DP): The model is replicated across different GPUs, each getting a slice of the data for parallel processing.

• Tensor Parallelism (TP): Individual layers of the model are partitioned across multiple devices, enabling intra-layer parallelism.

• Pipeline Parallelism (PP): Layers of the model are divided across different GPUs so that each device computes only a fraction of the layers, also known as vertical parallelism.

Efficiency:

By integrating these types of parallelism—often termed 3D parallelism—the framework allows BLOOM to scale to hundreds of GPUs with incredibly high utilization.

Result? A peak performance of 156 TFLOPs on A100 GPUs. In essence, BLOOM isn't just another LLM; it’s a monument to the power of collective genius.

BLOOM is available in the following versions: bloom-560m, bloom-1b1, bloom-1b7, bloom-3b, bloom-7b1, bloom (176B parameters).

Current Situation including Fundraising, Acquisitions, Current state by the numbers, as well as Business model and Revenue streams →

What exactly can you find on their platform?

If you want to start on the platform, here is a step-by-step guide.

Products & Services

• Transformers: Open-source NLP library, interoperable with various ML models and libraries. One of the most popular and advanced frameworks for Transformer architectures in deep learning applications.

• Hugging Face Open LLM Leaderboard

• Hugging Face Hub: Central repository for pre-trained ML models, datasets, collaborative spaces, and documentation.

• Tasks: A marketplace for ML and AI problems that users can tackle.

• Interface APIs: For users to serve models directly, simplifying the deployment process.

• Autotrain: Allows automatic training, evaluation, and deployment of ML models.

• Enterprise Hub/Private Hub: A private version of the Hugging Face Hub for enterprise users.

• Spaces: A platform for developers to host and share their ML models, with resource limitations for free users. Notable developer Spaces on the platform include DALLE-E mini by Dalle-mini, Stable Diffusion by Stabilityai, AnimeGANv2, and ArcaneGan by Akhaliq and Latent Diffusion by Multimodalart.

• Models that contain Tasks, Libraries, Datasets, Languages, Licenses and Others.

• Datasets Library: A vast repository of datasets that can be accessed programmatically.

• Daily Papers: Trending papers.

• Expert Support & MLOps Solutions: Ranging from customized support, training, fine-tuning, deployment services, optimizing facilities, APIs, and on-premise solutions for ultra-fast inference, no-code tools

So, in a nutshell, Hugging Face has wedged itself quite comfortably at the intersection of open-source idealism and pragmatic enterprise solutions, making it both an industry darling and a revenue magnet.

Founders’ attitude toward AI risks

Clement Delangue has a practical outlook on AI risks. His attention is less on the theoretical dangers of Artificial General Intelligence (AGI) and more on the immediate concerns.

Thomas Wolf is particularly concerned about the societal risks of AI monopolies. He points out the dangers of letting "an opaque monopoly or oligopoly appear on a technology as groundbreaking as AI.” Wolf envisages a future where such monopolistic AI could unduly influence personal and political decisions. His remedy is “to build a set of responsible, open, diverse and educated ecosystems around all new usages of AI.” He recommends reading the book «Overview of Catastrophic AI Risks» by Dan Hendricks.

Hugging Face has acknowledged the pervasive issue of biases in AI models, particularly in Natural Language Processing (NLP). They have taken steps to address this, such as implementing the Model Card feature, initially developed by Dr. Margaret Mitchell, a former Google ethicist, who Clement Delangue hired to focus on research related to bias and fairness.

Margaret Mitchell and her colleague Sasha Luccioni are very vocal about biases and fairness in ML and regularly share their thoughts and research on these topics.

They and other Hugging Face employees create a constant flow of educational materials and – what is probably even more important – constantly question the companies and their decisions regarding openness in ML.

Inclusivity and public education about AI are important to Delangue. He believes that “If users don’t understand how this technology is built, it creates a lot of risks, a lot of misconceptions.” His solution is fostering a community where anyone invested in the future of AI can engage, learn, and contribute.

Both Delangue and Wolf see the bigger picture of AI; they understand its immense potential and the ethical pitfalls that come with it. They advocate for regulation, transparency, and public education as antidotes to the most immediate risks and ethical concerns surrounding AI. Furthermore, they aim to make Hugging Face a platform that not only advances the technology but also engages in critical conversations around its ethical ramifications. Their combined message seems clear: With great computing power comes great responsibility. And that responsibility includes avoiding monopolistic control and addressing biases.

So “Everyone seems to love them. How did they achieve it?” Personal note from the editor: I've been scouring hundreds of articles, blogs, and tweets from major AI companies, their leaders, and teams, and as a specialist in linguistic analysis, I can only admire the consistency of the narrative with the actual actions. Their tone steadily strikes a balance—friendly, engaged, and concerned, yet devoid of doomism. Having observed Hugging Face from the outside for the past three years (and I am not associated with them anyhow and haven’t met any of the founders or employees), I'm struck by the company's cohesive culture and the unity among its employees across all social media platforms. They maintain sincerity and coherence in both their words and actions, a noteworthy trait given the current climate of uncertainty surrounding AI. I can only wish that more companies would adopt Hugging Face's attitude.

Regulations

Delangue is keen on regulation, especially for more transparency.

Instead of the Conclusion

In 2013, the year when Clement Delangue came to the US, Julien Chaumond wrote a blog post “The Quest for the Ideal CEO-type Co-Founder.”

Over 15 months, Julien met with over 30 CEO-type individuals from various backgrounds, looking for a complementary co-founder, primarily in Paris and London.

The Three Key Traits he was looking for:

• The Good Hustler: Effective hustle, not just talking big, but showing it through actions such as successful PR campaigns, sales skills, and a track record of launching startups, raising funds, or closing significant deals.

• Appreciation for Tech: Understanding the crucial role of technology in a tech startup, recognizing the co-founder's significant contribution, and not undermining its importance, especially if they have had successful exits before.

• Idea Alignment: Finding a co-founder with either a shared passion for a startup idea or one who can get passionate about your idea. This shouldn't be the sole or the most critical trait but should align with the goals of the startup.

The blog post received 12 likes and one comment:

He certainly did.

Bonus: All important links about the founders

To be continued…