Ignatius was brought to Greenwich as an orphan at the age of two. He was owned by three sisters who did not believe in the education of enslaved Africans. Ignatius taught himself to read and write. He developed his talents as a writer and a musician with the encouragement of his benefactor, the Duke of Montagu. When the Duke died, Ignatius ran away from the demanding sisters and worked as a butler for the Montagus in Blackheath.
He retired from domestic service due to ill health and opened a grocery shop on Charles Street in Westminister. Here he settled into family life with his Caribbean wife and two children, William and Elizabeth.