Mary Isabella Lee is shown with her son John A. Lee and grandson John Rowe. Mary's stormy relationship with her common-law husband Alfred had ended years before. Never formally married, the couple separated, came back together, and then separated permanently. Division of property was not an issue – they had little, and Mary accepted Alfred selling furniture they had received as a 'wedding' present – nor was custody of children. Mary Lee (who continued to use Alfred's surname) struggled to bring up her children on a dressmaker’s income, supplemented by charitable aid.

John A. Lee later wrote (at first anonymously) about his family in Children of the poor. His mother did not agree with his version of her life, insisting on her own respectability in her autobiography The not so poor, published decades after her death. John A. Lee was a notable member of the Labour government elected in 1936, and believed that, like him, several other members of the first Labour government were illegitimate.