Note/Moore's Law — AI Scaling Law

»The [AI] scaling law indicates that the performance of Large Language Models (LLMs) can continually improve as the model size increases« ¹; it is reminiscent of »Moore’s Law« ² in the context of computer architecture.

»Moore’s Law« (if we are honest, one ought to call it »Moore’s Observation« as it isn’t a physical law but rather an observation, reformulated as a prediction, that the number of transistors on a microchip doubles approximately every two years) is a frequently invoked prediction, strategically deployed to justify extraordinary capital investments into silicon manufacturing. Analogous to »Moore’s law,« AI scaling laws—a discussion frequently relabeled as »Large Language Models«—express a scaling »principle« that »more data« leads to »better« AI models and performance. In contemporary discourse about artificial intelligence, the »AI scaling principle« is exploited and deployed akin to »Moore’s Law,« i.e., as part of a »corporate propaganda« effort to encourage astronomical investments and distract from the gargantuan subsidies extracted from the public.