Alby James was born in north London. He obtained a BA in Development Studies at the University of East Anglia before turning to directing as a career. In 1979 he was appointed as Arts Council Trainee Assistant Director at the Royal Court Theatre. James was also Assistant Director at the Royal Shakespeare Company for its first season at the Barbican Centre. In 1984 he became Artistic Director of Temba Theatre Company.