Gamu Nhengu, who is originally from Zimbabwe, was controversially eliminated from the ITV show on Sunday.
Immigration officials have confirmed that her mother, Nokuthula Ngazana, had been refused permission to stay in the UK after her visa expired in August.
The 18-year-old’s family has lived in central Scotland for eight years.
A statement from the UK Border Agency said: “The applications made by Ms Ngazana and her family were considered in line with the published immigration rules.
“Ms Ngazana’s application was refused as it did not meet all of the conditions for approval.
“Her family, who had applied as her dependants, were therefore also refused.”
The family currently live in Tillicoultry, Clackmannanshire, where Gamu attended Alva Academy.
They were allowed to stay in the country while Ms Ngazana studied at university but the visa has now run out.
Officials confirmed they must now leave the UK voluntarily or face deportation to their native Zimbabwe.
Ngazana said she was not aware of UK Border Agency’s decision.
She refused to comment on her daughter’s elimination from the show.
A spokesman for the Home Office said the decision to axe Gamu from X Factor was the “business” of the television show.
He could not say whether the omission by judge Cheryl Cole had been decided after talks between immigration officials and the programme’s bosses.
The Home Office said Ngazana does not have a right to appeal and her singer daughter cannot apply separately to remain in this country.
More than 200,000 people have joined a Facebook campaign calling for the return of Gamu to the show. BBC