The state-of-the-art in generative AI is advancing fast. But, unlike previous AI waves marked by big launches and research papers, generative AI is spreading in a much more grassroots (and unlikely) medium: through Google Colab notebooks.
Google Colab notebooks are free Jupyter notebooks that run in the cloud and are easy to share. Many people use them to tinker with models, experiment with code, and share ideas. Interestingly, it was launched by Google Research during the time I worked on Google Cloud AI (we shipped a similar but unbranded Jupyter workflow).
So why are Colab notebooks the medium of exchange?
First, the base infrastructure and models are already open-sourced and developed. During the last wave, TensorFlow and PyTorch were still being incubated as solutions to the problems of deep learning. The biggest models were either closed-source or too complex for the average developer to contribute to.
This time, there’s a lot of “plumbing” work that’s being done in forked GitHub repositories that don’t require deep knowledge of machine learning or diffusion models. Those changes could be modifying Stable Diffusion code to run them on consumer M1 GPUs or creating Web UIs or user interfaces to run text2img or img2img and tune parameters. Or maybe it’s modifying the model to run in a different framework or with even fewer resources.
Second, LLMs are more consumer-friendly. Normal users and developers can make sense of the model. Inputs (prompts) and outputs (images) are more accessible to the average user than bounding boxes, vector embeddings, or NumPy arrays. Models are smaller and can be run on commodity hardware. Datasets are relatively small, or trained weights are published.
Third, diffusion models are goldilocks models for Colab — too large to fine-tune or run inference on the average laptop but small enough to run on spot instances that are given away for free.
There are some interesting implications of Colab as a medium that ML applications go viral on:
- Security — most of these models download and run code from GitHub. They might ask for permission to access your Google Drive. It isn’t easy to know exactly what’s going on in a notebook, and there are few guarantees that it’s doing what you think it is.
- Presentation and code — A cardinal programming rule is separating presentation and code. But sometimes it’s helpful to combine the two. I wrote about this in Presentation Next to Code and In Defense of the Jupyter Notebook.
- Monetization – Colab is unlikely to drive real infrastructure spend for cloud. While some consumers might pay for Colab Pro+ ($50/mo), it doesn't seem like a real business model (is it Enterprise SaaS? Does it belong in the same category as Google Workspace Docs/Sheets/Mail?). Google can subsidize Colab through other products, but in the long run, it should be self-sustainable. Maybe it follows a Hugging Face-like playbook (although it's unclear exactly what the end-result looks like even in that case).