In 2009, The Hold Steady began writing Heaven Is Whenever while touring in support of their previous album, Stay Positive. According to guitarist Tad Kubler, the band "turned the back lounge of the [tour] bus into a make shift vocal booth where we could also overdub guitars. These songs gradually became the demos for the new record, and we continued writing. It allowed us to work at a pace that was dictated almost solely by our creative output. We just hit record whenever we thought we had something worth putting down."

The band subsequently entered the studio with Dean Baltulonis, producer of the band's debut album, Almost Killed Me, and its follow-up Separation Sunday. Vocalist and guitarist Craig Finn states that Baltulonis is "a real close friend, so it was kind of a more relaxed, less formal way of making a record. We kind of went in for shorter sessions over a longer period of time, and we recorded a lot. There’s a fair amount of songs that didn’t make the record, and that’s always a heartbreaking thing, choosing between your favorite children."

1. The Sweet Part of the City
2. Soft in the Center
3. The Weekenders
4. The Smidge
5. Rock Problems
6. We Can Get Together
7. Hurricane J
8. Barely Breathing
9. Our Whole Lives
10. A Slight Discomfort