An inquiry into Downing Street lockdown parties should be led by an independent figure, opposition MPs have said.
The UK's top civil servant, Simon Case, had been chosen to lead the investigation but he stepped aside after it emerged an event was held in his own office last December.
No 10 said the probe would be concluded by senior civil servant Sue Gray.
But Labour MP Chris Bryant and the SNP's Westminster leader questioned whether she could be independent.
Mr Bryant, who is chairman of the Commons standards committee, said he was not convinced about Ms Gray leading the investigation.
"She will know every single one of them [involved in the parties] I would have thought, so again I'm not sure that this is the right direction," he told BBC Radio 4's Today programme.
- Kuenssberg: PM has been put on notice by his own side
- Which parties are being investigated?
- PM urged to get a grip after by-election rout
- Department for Transport sorry for Christmas party
The SNP's Ian Blackford suggested there should be a judge-led inquiry.
He told BBC Two's Newsnight that "having somebody else from the civil service marking their own isn't good enough" and that it should be someone with authority from outside the government and civil service.
However, Lord Barwell, a former Conservative MP who was chief of staff for then-Prime Minister Theresa May, said he could not think of a better person for the role.
In past investigations he said Ms Gray had been "completely independent" and he was confident "we will get the facts".
Labour's deputy leader Angela Rayner said Ms Gray had an "incredible responsibility to restore trust".
Ms Gray, who previously served as head of the ethics team in the Cabinet Office, is the second permanent secretary at the Department for Levelling Up, Housing and Communities.
A spokesperson for No 10 said: "To ensure the ongoing investigation retains public confidence the cabinet secretary [Simon Case] has recused himself for the remainder of the process."
This investigation was supposed to clear up whether rules had been broken – and many hoped it would allow the government to move on.
Instead, it rumbles on and on. We are seeing more allegations – right up to the office of the man who was given the job of investigating.
It is deeply embarrassing for the government – and for the cabinet secretary – that he is now seen to have a conflict of interest in this inquiry.
The big picture is that Boris Johnson has faced a torrid few weeks which have left his party jittery.
The list is long; the handling of sleaze allegations, the record Tory rebellion in the Commons, the loss of an incredibly safe seat, the continued fallout from allegations of Christmas parties in Whitehall.
Christmas cheer will be in short supply in the Conservative Party this year. And many MPs will be hoping the prime minister makes some significant new year's resolutions to steady the ship.
The BBC understands a quiz was held for members of Mr Case's private office on 17 December 2020, and invites were sent out titled "Christmas Party!"
About 15 people were invited to the gathering at 70 Whitehall, near Downing Street, although it is understood not everyone turned up.
Responding to the claims, a government spokesperson said: "Staff in the cabinet secretary's private office took part in a virtual quiz on 17 December 2020.
"A small number of them, who had been working in the office throughout the pandemic and on duty that day, took part from their desks, while the rest of the team were virtual.
"The cabinet secretary played no part in the event, but walked through the team's office on the way to his own office.
"No outside guests or other staff were invited or present. This lasted for an hour and drinks and snacks were bought by those attending. He also spoke briefly to staff in the office before leaving."
The party was first reported on the Guido Fawkes website on Friday afternoon.
Mr Case, who has been cabinet secretary since last September, had been asked by Prime Minister Boris Johnson to investigate gatherings for staff in government buildings last year, while the country was under Covid restrictions.
His inquiry was focusing on events in Downing Street on 27 November, 15 December and 18 December, and at the education department on 10 December.
Who is Sue Gray?
The new head of the inquiry into government lockdown parties has worked in several departments, including transport and health.
But she is not exactly a typical career civil servant.
In the late 1980s she took a career break to run a pub called the Cove Bar in Northern Ireland.
Speaking to the BBC she said: "I loved it, loved it at the time, I'd never do it again."
Having served as head of the ethics team in the Cabinet Office, she has experience of investigating ministers, officials and advisers.
In 2012, her inquiry into "plebgate" led to the resignation of then-chief whip Andrew Mitchell. Mr Mitchell admitted using bad language towards a Downing Street police officer but denied calling him a "pleb".
And in 2017, one of her investigations forced Damian Green to step down as a minister after he was found to have made "inaccurate" statements over what he knew about claims pornography was found on his office computer in 2008.
Last year she applied to become head of the Northern Ireland Civil Service but was rejected – something she attributed to being "too much of a challenger".
She is married to a country and western singer, originally from County Down, called Bill Conlon.
Read a BBC interview with Ms Gray here.