Number 10 has said adviser Andrew Sabisky has quit following criticism of alleged past racist and sexist remarks on pregnancies, eugenics and race.
Labour had called for Sabisky to be fired for reportedly saying black people had lower average IQs than white people.
The latest controversy has been condemned by Tory MPs who have called Sabisky's comments and called for a review into how aides are recruited.
Business minister Kwasi Kwarteng has now described the remarks as "offensive" and "racist", despite the Prime Minister refusing to issue a condemnation over the appointment.
Mr Kwarteng added that he thought the Government would be "looking at vetting processes more closely" in light of the drama.
The business minister also told BBC Radio 5 live he was glad the adviser had left.
He said the government “should prevent racists from coming into No 10 or wherever he was working”, adding: “The fact is, his remarks have been identified and very quickly pushed out … and we can move on.”
Downing Street has so far declined to give even basic details of Sabisky’s work, or how he was recruited. Officials also refused to comment on the status of two other so-called “super-forecasters” pictured with him outside No 10 in January.
Mr Sabisky tweeted: "I wanted to help the government not be a distraction... accordingly I've decided to resign." "I know this will disappoint a lot of people but I signed up to do real work, not be in the middle of a giant character assassination," it continued. "If I can't do the work properly there's no point, and I have a lot of other things to do with my life."
Mr Sabisky, who describes himself as a "superforecaster", was appointed earlier this year after the prime minister's chief adviser Dominic Cummings called for "misfits and weirdos" to apply for jobs in Downing Street.
Leaving his home on Tuesday, Mr Cummings told reporters: "Read Philip Tetlock's
Superforecasters, instead of political pundits who don't know what they're talking about."
The reference appeared to be to a 2015 book of similar name on the science of prediction, Superforecasting, co-written by US-based academic Philip Tetlock.
American psychologist Philip Tetlock came up with the Good Judgment Project as part of a US government competition to find better ways of predicting. He looked at thousands of predictions by experts and found they were no better than if they had selected outcomes at random, which he compared to chimps throwing darts at a board.
Prof Tetlock then asked thousands of people to come up with figures for the chances of a range of things happening, such as a nuclear test by North Korea in the next three months.
A few months later, he selected the most successful of the forecasters - and found, in later exercises, they continued to make better predictions even than those in the intelligence services who had access to secret information.
As news of Sabisky’s role emerged, so did details of views he had expounded energetically over years of writing, blogging and social media comments, much of it based around a belief in genetics as the main driver for people’s life chances.
It prompted a handful of Conservative MPs to speak out publicly against Sabisky, with other government advisers left notably unhappy at the coverage. One said: “We don’t want to all be tarred with the same brush – that he is in some way representative of all spads.” Downing Street had appeared resolute in sticking by Sabisky throughout Monday, but he abruptly left that evening amid widespread reports that he had been ordered to step down.