'It was only in later years, after the death of our father and then our mother, that we became closer'