Ethologists describing animals in the wild have long appreciated that naturalistic, self-motivated behavior is built from modules that are linked together over time into predictable sequences. Many such sequences are built to extract information from the environment. And yet, it remains unclear how the brain regulates the selection of individual behavioral modules for expression at any given moment, or how it dynamically composes these modules into the fluid behaviors observed when animals act of their own volition, and in the absence of experimental restraint, task structure or explicit reward. Here we use novel methods for characterizing spontaneous mouse behavior — combined with neural recordings and closed-loop manipulations — to reveal mechanisms used by the brain to create the architecture of self-guided behavior. I will describe recent results in which we explain behavioral variation across timescales ranging from milliseconds to millennia.