Why?

• Plausibly some protection against retaliation. However, most pseudonymous writing was quickly attributed to authors.
• Power in names. The names weren’t chosen at random. Often, they called back to famous Romans who took part in the formation of the Roman Republic. Or others who were known for their virtue or principles.

Alexander Hamilton might have written under the most pseudonyms (at least five). Benjamin Franklin used at least three. Here’s a list of some of the more popular ones around the time of the American Revolution.

Phocion (Alexander Hamilton) — Essays defending the Jay Treaty with Great Britain. Phocion was an Athenian statesman known for his integrity and opposition to demagoguery.

Columbus (Alexander Hamilton) — Defending the Continental Congress and criticizing British policies.

Publius (Alexander Hamilton, James Madison, John Jay) — The authors of the Federalist Papers, which were a series of essays advocating for the ratification of the Constitution. Individual authorship wasn’t released until Hamilton’s death, and even then historians are still trying to match authors to text. It’s hypothesized that Hamilton wrote 51 essays, Madison 29, and Jay 5. Publius Valerius Poplicola was a Roman consul known for his role in founding the Roman Republic.

Historicus (Alexander Hamilton) — Essays on various topics related to the Constitution and federalism.

Pacificus (Alexander Hamilton) — Used to defend President George Washington's Neutrality Proclamation of 1793 (declared the U.S. neutral in the conflict between France and Great Britain). “Making peace” in Latin.

Helvidius (James Madison) — Written in response to Pacificus (Hamilton), these essays defended the constitutional authority of Congress in foreign affairs. Helvidius Priscus was a Roman senator known for his defense of republicanism and freedom of speech.

Americanus (John Jay, John Stevens, Jr.) — Federalists essays.

Candidus (Benjamin Franklin) — Writings advocating for various causes, including opposition to oppressive British policies.

Silence Dogood (Benjamin Franklin) — A fictitious widow created by Franklin to offer social commentary.

Richard Saunders “Poor Richard” (Benjamin Franklin) — Used to publish Poor Richard’s Almanack. The name comes from a popular London almanac, Rider’s British Merlin.

“Common Sense” — Thomas Paine’s pamphlet advocating for American independence was initially published anonymously.

Cincinnatus (Arthur Lee) — Anti-federalist papers.

A Farmer (John Dickinson) — Essays titled "Letters from a Farmer in Pennsylvania," which argued against the Townshend Acts imposed by the British.

Cato (George Clinton) — Anti-federalist essays around the time of the ratification of the Constitution. Attributed to George Clinton, but not confirmed. Cato the Younger was a Roman statesman known for his staunch republicanism and opposition to Julius Caesar.

Brutus (Robert Yates) — An ally of George Clinton’s who wrote more anti-federalist essays. Marcus Junius Brutus was a Roman senator famous for his role in the assassination of Julius Caesar, symbolizing resistance to tyranny.

Centinel (Samuel Bryan) — A series of anti-federalist essays critical of the proposed U.S. Constitution's centralizing tendencies.

Americanus (John Stevens, Jr.) — Essays written to support the Federalist cause and the ratification of the U.S. Constitution.

Poplicola (John Adams) — Essays defending the British constitution and criticizing the Stamp Act. The same Publius Valerius Poplicola used by Hamilton.

Novanglus (John Adams) — A series of essays written in response to Massachusettensis, defending colonial rights. Latinization of “New Englander”.

A Citizen of New York (Martin Van Buren) — political essays.

But Shockley was a terrible manager, and the students left to form their own company the next year, Fairchild Semiconductor. They would be later known as the “traitorous eight”.

The founders of Fairchild Semiconductor were: Gordon Moore, C. Sheldon Roberts, Eugene Kleiner, Robert Noyce, Victor Grinich, Julius Blank, Jean Hoerni, and Jay Last.

Fairchild Semiconductor became the proto-company of Silicon Valley. Many major technology companies can somehow trace their founding or story to Fairchild.

Intel - Founded by Robert Noyce and Gordon Moore, both former employees of Fairchild Semiconductor.

AMD (Advanced Micro Devices) - Founded by Jerry Sanders, another Fairchild alumnus.

Kleiner Perkins - A venture capital firm co-founded by Eugene Kleiner, a former Fairchild employee.

Sequoia Capital— Don Valentine worked at Fairchild Semiconductor for seven years before moving to National Semiconductor (another Fairchild). Then, he started Sequoia Capital.

Other companies founded by Fairchild employees: SanDisk, National Semiconductor, Altera, LSI Logic, Amelco, Applied Materials, and more.

• Hundreds of open-source models. Varying sized models from small to very large. Many are chat-tuned similar to ChatGPT.
• Distilled models from ChatGPT. Academics and competitors both used data from ChatGPT conversations to train or fine-tune their own models.
• Competition. Microsoft launched Bing Chat. Google launched Bard. Poe, Pi, Perplexity. Claude by Anthropic. Not to mention self-hosted open-source chat UIs and other wrappers. There’s no shortage of competition (although ChatGPT still is the most popular).
• RAG is hard. “Browse with Bing” and Bing Chat launched but hallucinations are still an issue. Browsing the internet doesn’t seem like the catch-all
• Not every launch increased performance across the board. Every new iteration of ChatGPT launched changed the way the model behaved. Many queries got better. Some got worse. Google has always had this problem as well, but applications aren’t build on Google.
• A consumer subscription model. ChatGPT Plus was released in February 2023. The consumer model maybe competes with the developer and enterprise products (why not just use the API?).
• Multi-modal. ChatGPT started to accept images and files in the chat. DALL-E and the vision API became integrated into the chat window. There are open-source models that are multi-modal, but so far no experience is as sleek as OpenAI’s.
• Plugins launched but never found product-market fit. Plugins launched but didn’t become the App Store that OpenAI hoped. Custom GPTs seem to be the next strategy for extensibility, although they won’t launch until next year.
• Code Interpreter is getting better. Agents and tool-use is still hard for LLMs. But it’s getting better and becoming more useful. Files can now be added directly to the UI to chat with.

The fallacy usually follows the same four steps.

1. Measure what can easily be measured.
2. Dismiss what can’t be measured easily.
3. Presume what can’t be measured easily isn’t important.
4. Extrapolate and conclude that what can’t be measured doesn’t exist.

You can find the McNamara Fallacy in all types of disciplines. The emphasis on standardized tests in education (at the expense of less quantifiable qualities and learning). Or when the success of treatments in medicine is based only on easy to measure outcomes (not quality of life, mental health, or overall well-being). Or optimizing for short-term financial metrics at the expense of brand reputation, employee satisfaction, or other intangibles.

To train LLMs, you need data that is:

1. Large — Sufficiently large LMs require trillions of tokens.
2. Clean — Noisy data reduces performance.
3. Diverse — Data should come from different sources and different knowledge bases.

What does clean data look like?

You can de-duplicate data with simple heuristics. The most basic would be removing any exact duplicates at the document, paragraph, or line level. More advanced versions might look at the data semantically, figuring out what data should be omitted because it’s better represented with higher quality data.

The other dimension of clean data is converting various file types to something easily consumed by the LLM, usually markdown. That’s why we’ve seen projects like nougat and donut convert PDFs, books, and LaTeX to better formats for LLMs. There’s a lot of training data that’s still stuck in PDFs and human-readable but not so easily machine-readable data.

Where does diverse data come from?

The surprising result of the success of the GPTs is that web text from the Internet is probably one of the most diverse datasets out there. It contains usage and data that aren’t found in many other data corpora. That’s why models tend to perform so much better when they’re given more data from the web.

Why is Discord such a good GTM for AI applications?

• Text interface. Most users are just generating images, videos, and audio in these Discord servers. Prompts are easily expressible in simple text commands. It’s why we’ve seen image generation strategies like Midjourney (all-in-one) flourish in Discord while more raw diffusion models haven’t grown as quickly (e.g., Stable Diffusion with many configurable parameters).
• Virality. Prompt engineering models is difficult and more art than science (today). Users can see generations by other users and collectively see what’s working and what isn’t. This means that these communities often have the most advanced prompts and best images.
• Low friction. Go to where your users already are. Most developers have Discord now. One fewer application to sign up for.
• Free hosting. Discord pays for the image hosting and bandwidth. At Midjourney scale, this is not negligible.

But Discord has it’s risks as a platform to build on.

• Platform risk. Discord could (easily?) build its own Midjourney-type application into the platform. Using all of the prompt-image pairs (along with reactions as a RLHF), it could probably distill a much better model from Midjourney (questionably legal but technically easy). This reminds me of the Zynga / Facebook relationship. Zynga accounted for 19% of Facebook’s revenue at one point. Facebook reduced Zynga’s API access and launched its own gaming platform.
• Multi-modal. How does multi-modal fit into the Discord text-first interface? Sure there are images and audio that can be uploaded via the interface, but it’s hard to image the UI that a multi-modal AI will need in the future.

1. Underestimation of Incentives: Despite understanding the significant influence of incentives (reinforcement in psychology and incentives in economics), there's a tendency to consistently underestimate their power.
2. Psychological Denial: This is the refusal to accept reality because it is too painful or difficult to bear.
3. Incentive-Cause Bias: This occurs when personal incentives or those of a trusted advisor create a conflict of interest, leading to biased decisions.
4. Bias from Consistency and Commitment: This involves a strong tendency to stick to pre-existing beliefs or commitments, even in the face of contradictory evidence.
5. Bias from Pavlovian Association: This bias refers to the error of basing decisions on past associations or correlations without considering their current relevance or accuracy.
6. Bias from Reciprocation Tendency: This bias involves a natural inclination to reciprocate actions and behaviors, including conforming to others' expectations, especially when one is experiencing success or is 'on a roll.'
7. Bias from Over-Influence by Social Proof: This bias refers to the heavy reliance on the actions or decisions of others, especially in situations of uncertainty or stress.
8. Bias from Favoring Elegance over Practicality in Theory: This bias involves a preference for theories or explanations that are mathematically elegant or intellectually satisfying, even if they are less accurate in practical terms. “Better to be roughly right than precisely wrong” — Keynes.
9. Bias from Contrast-Induced Distortions: This bias refers to the way our perceptions, sensations, and cognition can be significantly altered by contrasts.
10. Bias from Over-Influence by Authority: This bias involves the tendency to conform to instructions or opinions provided by an authority figure, even when these instructions conflict with one's own moral judgment or common sense.
11. Bias from Deprival Super Reaction Syndrome: This bias is characterized by an intense reaction to losing or the threat of losing something, especially something that one perceives as almost possessed but never fully owned.
12. Bias from Deprival Super Reaction Syndrome: This bias is characterized by an intense reaction to losing or the threat of losing something, especially something that one perceives as almost possessed but never fully owned.
13. Bias from Envy/Jealousy: This bias stems from feelings of envy or jealousy towards others.
14. Bias from Chemical Dependency: This bias relates to the cognitive and behavioral changes that result from chemical dependency, such as addiction to drugs or alcohol.
15. Bias from Gambling Compulsion: This bias refers to the compulsive urge to gamble, driven by the psychological principle of variable reinforcement.
16. Bias from Liking Distortion: This bias involves a preference for things that are familiar or similar to oneself, including one's own ideas, kind, and identity.
17. Bias from Disliking Distortion: This is the opposite of liking distortion, where there's a tendency to reject or not learn from sources that are disliked.
18. Bias from the Non-Mathematical Nature of the Human Brain in Probability Assessment: This bias refers to the human brain's tendency to rely on crude heuristics and be easily misled by contrasts when dealing with probabilities, rather than using precise mathematical approaches.
19. Bias from Over-Influence by Extra Vivid Evidence: This bias describes the tendency to give disproportionate weight to particularly vivid or emotionally striking information when making decisions.
20. Stress-induced mental changes, small and large, temporary and permanent.
21. Mental Confusion from Poorly Structured Information and Inadequate Explanations: This bias involves difficulties in understanding or decision-making due to information that is not well-organized or lacks a coherent theoretical framework.

There are many types of Monte Carlo Methods, but they all follow a general pattern — using random sampling to model complex systems.

A simple example: Imagine a complex shape you want to know the area of.

1. Place the shape on a dartboard.
2. Randomly throw darts at the dartboard.
3. Count the number of darts that are inside the shape and outside.
4. The estimated area of the shape is = (number of darts in shape / number of darts outside of shape) * the area of the dartboard.

(This is computing a definite integral numerically with a method that doesn’t depend on the dimensions! You can even easily estimate the error given the number of samples).

Monte Carlo Tree Search (MCTS). Or use it to play a game like Blackjack (Chess, Go, Scrabble, and many other turn-based games) with Monte Carlo Tree Search. AlphaGo and its predecessors (AlphaGo Zero and AlphaZero) used versions of Monte Carlo Tree Search with reinforcement learning and deep learning.

The idea is fairly simple — add a policy (i.e., a strategy to follow) to the random sampling process. You might start with a simple one (random or stay with a hand under 18). For every move in a game, add that to a tree that describes the game. For Blackjack, that might be a series of hits or stays. When a game is won or lost, go back and update all of the nodes in the tree for that game (the “back propagation”).

After many games, you have a tree of expected utility for each move — that means you can sample the next move much more effectively. The value says something like — “given this current hand and set of actions, I won X% of the time”. You can get more advanced with the reward and update function — for example, you might discount wins that take many turns and prioritize quicker wins.

The business model: sell one item at break-even or for free to increase the sales of the complementary good. This is the “razor and blades” model. (Despite being named after the safety razor industry, early companies like Gillette didn’t initially follow this model).

This model works especially well when there are switching costs or vendor-lock in. If there are no switching costs, other providers can come in and compete margins away from the complementary good. When the K-cup patent expired in 2012, prices came down when competitors started producing compatible pods.

Or when a producer owns a monopoly on the complementary good. John D. Rockefeller and Standard Oil gave away eight million kerosene lamps. Demand for kerosene (conveniently sold by Standard Oil) skyrocketed.

Some other examples of the razor and blades model:

• Kindle e-reader / digital books.
• Video game console / video games
• Mobile phone / cellular data plan
• Electric toothbrush / replacement brush heads
• Printers / ink cartridges
• E-cigarettes / e-cigarette pods

The problem: databases are still regional. If you move your application logic closer to the user via edge functions in multiple regions, this most likely increases the distance between your application and your database. Since the latter is often more chatty (more data sent back and forth between the application and database than the user and the application), this usually increases latency.

Could you make data multi-regional? Sort of. There’s so work being done to bring the database to the edge (see distributed SQLite), but now with stateful data at the edge, you have a complicated distributed systems problem.

Smarter caching? There’s also some work being done in application frameworks to do smarter caching (e.g., stale-while-revalidate) so that users get fast responses for most of the application while new data is rehydrated.