Frontman Mukai Shutoku founded what would later be called Zazen Boys shortly after the demise of his previous band Number Girl. He initially recruited former Number Girl drummer Ahito Inazawa. Mukai's original idea for Zazen Boys centered around himself and Inazawa playing together with guest talent. This changed after Mukai brought in Hinata Hidekazu (ex-ART-SCHOOL) on bass and Yoshikane Sou (Kicking the Lion) to fill out the band's lineup.
He named the group Zazen Boys after an old project with 54-71 and pop singer hal. The band's self-titled album was released in January of 2004 to considerable underground success.
The band's debut continued the steady evolution apparent in Number Girl's final albums, but taken several steps further. The precise rhythms and minor-key melodies of Mukai's former band were still part of the equation, though a heavy dose of experimentation moved the sound even farther left-of-center.
Mukai started working on a new album the same year, released in September 2004. Zazen Boys II smoothed out the rough edges of the band's debut and featured another hometown friend, Shiina Ringo, on backup vocals.
Inazawa announced his sudden departure the next year. Mukai quickly brought in Matsushita Atsushi as the new drummer, then released the five-track single Himitsu Girl's Top Secret in July followed by Zazen Boys III in January 2006. The new album looked to be a push towards free jazz, with a clear focus on improvisation and ambient experimentation.