Piecewise

May 20, 2023

Opportunities multiply as they are seized.

Sometimes you have to think step by step. Executing a complete plan is hard enough, but it might be impossible — unknown constraints, path dependence, or other exogenous factors. Unapproachable problems sometimes become excruciatingly obvious when a subproblem is solved. It’s part of why so many great scientists have had an annus mirabilis (“miracle year”), where they develop multiple groundbreaking theories. Einstein wrote four papers in 1905 (about Brownian motion and special relativity). Newton developed calculus and the theory of gravity in 1665.

What seems like a giant leap is sometimes just a collection of piecewise developments (albeit ones that no one had yet reconciled). Maybe that’s why we often have simultaneous but independent inventions (Leibniz may have also invented calculus around the exact same time as Newton).

Solving step by step lets us efficiently debug and diagnose problems and compartmentalize the effort.

Language models tend to think better step by step as well. If you ask them to write out their reasoning, they tend to perform better on tasks. Is piecewise a universal law? Or are these models just reflecting a human preference?