The report found that AI-generated scams are making it harder for people to tell the difference between real and fake content, with a trebling in the number of disputes reported this year, compared to last.
Facebook, Google Maps, Instagram, Pinterest, Threads, TikTok and YouTube were all platforms examined in this report, following disputes reported by members of the public across the EU.
The reports are sent to Appeals Centre Europe, an independent, out-of-court dispute settlement body, headquartered in Dublin. It was set up under Article 21 of the EU's Digital Services Act.
Appeals Centre Europe is working with Coimisiún na Meán and the European Commission to try to ensure that its decisions are implemented by the platforms it covers.
Its decisions are non-binding, but platforms must engage with the organisation in good faith.
Appeals Centre Europe has received over 65,000 disputes from people and organisations across the EU over decisions made by these social media platforms.
The volume of disputes has resulted in a doubling of its Dublin-based review team, with five additional staff hired in recent months and more vacancies now being advertised.
The report found that social media platforms often break their own rules by leaving up scam content.
In some cases, platforms had left up content that was violent or graphic and content that contained hate speech or that was linked to dangerous or criminal organisations.
Some 43,000 of the disputes examined arose during the six-month period on which this new report is based, from April to September of this year.
That amounted to an average of one dispute every 12 minutes, or roughly 200 disputes each day.
More than 19,000 of these reports were found to be eligible for investigation by the centre, with the majority of complaints coming from France, followed by Belgium and Spain.
Examples of the problems reported included an AI-generated 'romance scam' on TikTok, with a video of a woman saying that she is looking for a committed relationship, with the goal of getting people to send her money.
TikTok videos claiming the Valencia flash floods had been caused by climate engineering, and an AI-generated video about the Russia-Ukraine war were also found to have breached TikTok’s own rules on misinformation.
A video on Facebook was found to be an AI-generated deepfake of an economic expert from Moldova, promising viewers a 2,000% return on their investment.
A YouTube video about cryptocurrency told people to copy a ChatGPT code using a link in the caption, which the report found could have given the would-be scammer access to the viewer's crypto account.
Racist comments about black football players were found to have been left up on Instagram following a Champions League game.
In three-quarters of cases, where it was able to review content, Appeals Centre Europe overturned decisions which had been made by the platforms themselves. In the case of YouTube it overturned 93% of their decisions.
It also disagreed with TikTok 77% of the time, Instagram 75% of the time and with 67% of Facebook's decisions. It received only 17 eligible disputes abut Pinterest.
The organisation received three times the number of disputes in September this year, compared to the same time (October) last year.
'Social media should not be a safe haven for scammers'
Commenting on the report, CEO of Appeals Centre Europe Thomas Hughes said the growing number of disputes showed a clear appetite from the public to challenge decisions made by social media companies:
"From stealing people’s savings to impacting their mental health, scams destroy lives ... Social media should not be a safe haven for scammers." he said.
"People using social media must have the option to challenge platforms when they fail to enforce their own rules on fraud and scams."
Hacked accounts, copyright disputes and child sexual abuse materials were outside the scope of the report.
Many disputes over account suspensions also fell outside its remit, but it was able to give over 8,500 decisions in this area, overturning decisions to suspend social media users' accounts in roughly a quarter of cases across all platforms.
The centre says that, in the vast majority of cases, platforms are still not sharing the content that led to the suspension, making it harder for them to produce an independent review.
This was most pronounced with Google, where despite requesting more than 850 pieces of content between October of last year and September of this year, mainly about YouTube cases, it only received two.
It said Google has recently said it would share more content related to pending disputes, although it contended that online comments were outside the scope of the Digital Services Act.
Two offences to be added to legislation
Acting Minister for Communications Peter Burke said that two additional strike offences would be added to the AI act.
These will be in relation to non-consensual sexual content put online and child sexual abuse material generated by AI, he said.
Mr Burke added that he is looking at how social media companies are keeping up with their responsibilities.
"As a Government, we are saying very firmly to social media companies now that penalties are going to be very, very strict. This will not be tolerated in a fair society, and you have responsibilities."
Mr Burke said that, due to a lack of human moderators, content has remained on social media after being reported.
He said that gardaí are working on bringing people to justice after non-consensual sexual imagery was shared on X earlier this year.
"They believe they have the power," he said.
Mr Burke added that the Government wants to protect the most vulnerable of "our children" in relation to the generation of material.
Additional reporting Róisín Cullen




