Millions of workers face having to give money back to the Government after inadvertently paying the wrong amount of tax.

Mistakes by HM Revenue and Customs (HMRC) left an estimated 5.5 million underpaying or overpaying their tax through the Pay As You Earn System (PAYE) in 2013-14, according to the Daily Telegraph.

Around 3.5 million people are thought to have paid too little and will now have to pay the money back over the coming years. A further two million paid too much tax and will be able to claim it back from the Government.

Tax officials estimate the average error is around £300, the newspaper said.

The 5.5 million errors for 2013-14 are higher than the 5.2 million in the previous year, despite the introduction of a £270 million scheme designed to make the tax system more accurate.

The PAYE system checks tax paid against tax owed at the end of each year, meaning under- or overpayments can come about through changes in financial circumstances such as changing jobs or receiving benefits.

The HMRC's new Real Time Information (RTI) programme allows employers to report wage changes on a weekly or monthly basis, theoretically making tax payments more accurate.

But accountants said the increase in errors suggested the system was not working as promised.

David Heaton of Baker Tilly, told the Telegraph: "RTI was supposed to make PAYE more accurate, not less. So why are there more (errors) this year, with RTI in full flow, than last year, when RTI was only a pilot? The number of PAYE differences has risen, not fallen. Something in RTI is not working."

A spokesman for HMRC said that a staged roll-out of RTI meant some employers had not used they system for the full tax year, but early indications suggest tax processing was "in line" with expectations.

He said: "The effect of Real Time Information is not reflected yet as it has not bedded in but, over time, RTI will help to reduce the number of cases that have to be reconciled."