Some carers have been told to pay back thousands of pounds after being overpaid the benefit for years.