As digital platforms multiply and user-generated content expands very quickly, the need for effective harmful content detection becomes paramount. What once relied solely on the diligence of human moderators has given way to agile, AI-powered tools reshaping how communities and organisations manage toxic behaviours in words and visuals.

From moderators to machines: A brief history

Early days of content moderation saw human teams tasked with combing through vast amounts of user-submitted materials – flagging hate speech, misinformation, explicit content, and manipulated images.

While human insight brought valuable context and empathy, the sheer volume of submissions naturally outstripped what manual oversight could manage. Burnout among moderators also raised serious concerns. The result was delayed interventions, inconsistent judgment, and myriad harmful messages left unchecked.

The rise of automated detection

To address scale and consistency, early stages of automated detection software surfaced – chiefly, keyword filters and naïve algorithms. These could scan quickly for certain banned terms or suspicious phrases, offering some respite for moderation teams.

However, contextless automation brought new challenges: benign messages were sometimes mistaken for malicious ones due to crude word-matching, and evolving slang frequently bypassed protection.

AI and the next frontier in harmful content detection

Artificial intelligence changed this field. Using deep learning, machine learning, and neural networks, AI-powered systems now process vast and diverse streams of data with previously impossible nuance.

Rather than just flagging keywords, algorithms can detect intent, tone, and emergent abuse patterns.

Textual harmful content detection

Among the most pressing concerns are harmful or abusive messages on social networks, forums, and chats.

Modern solutions, like the AI-powered hate speech detector developed by Vinish Kapoor, demonstrate how free, online tools have democratised access to reliable content moderation.

The platform allows anyone to analyse a string of text for hate speech, harassment, violence, and other manifestations of online toxicity instantly – without technical know-how, subscriptions, or concern for privacy breaches. Such a detector moves beyond outdated keyword alarms by evaluating semantic meaning and context, so reducing false positives and highlighting sophisticated or coded abusive language drastically. The detection process adapts as internet linguistics evolve.

Ensuring visual authenticity: AI in image review

It’s not just text that requires vigilance. Images, widely shared on news feeds and messaging apps, pose unique risks: manipulated visuals often aim to misguide audiences or propagate conflict.

AI-creators now offer robust tools for image anomaly detection. Here, AI algorithms scan for inconsistencies like noise patterns, flawed shadows, distorted perspective, or mismatches between content layers – common signals of editing or manufacture.

The offerings stand out not only for accuracy but for sheer accessibility. Their completely free resources, overcome lack of technical requirements, and offer a privacy-centric approach that allows hobbyists, journalists, educators, and analysts to safeguard image integrity with remarkable simplicity.

Benefits of contemporary AI-powered detection tools

Modern AI solutions introduce vital advantages into the field:

• Instant analysis at scale: Millions of messages and media items can be scrutinized in seconds, vastly outpacing human moderation speeds.

• Contextual accuracy: By examining intent and latent meaning, AI-based content moderation vastly reduces wrongful flagging and adapts to shifting online trends.

• Data privacy assurance: With tools promising that neither text nor images are stored, users can check sensitive materials confidently.

• User-friendliness: Many tools require nothing more than scrolling to a website and pasting in text or uploading an image.

The evolution continues: What’s next for harmful content detection?

The future of digital safety likely hinges on greater collaboration between intelligent automation and skilled human input.

As AI models learn from more nuanced examples, their ability to curb emergent forms of harm will expand. Yet human oversight remains essential for sensitive cases demanding empathy, ethics, and social understanding.

With open, free solutions widely available and enhanced by privacy-first models, everyone from educators to business owners now possesses the tools to protect digital exchanges at scale – whether safeguarding group chats, user forums, comment threads, or email chains.

Conclusion

Harmful content detection has evolved dramatically – from slow, error-prone manual reviews to instantaneous, sophisticated, and privacy-conscious AI.

Today’s innovations strike a balance between broad coverage, real-time intervention, and accessibility, reinforcing the idea that safer, more positive digital environments are in everyone’s reach – no matter their technical background or budget.

(Image source: Pexels)

The post The evolution of harmful content detection: Manual moderation to AI appeared first on AI News.

This change was observed in the latest update to the Baidu Baike robots.txt file, which denies access to Googlebot and Bingbot crawlers.

According to the Wayback Machine, the change took place on August 8. Previously, Google and Bing search engines were allowed to index Baidu Baike’s central repository, which includes almost 30 million entries, although some target subdomains on the website were restricted.

This action by Baidu comes amid increasing demand for large datasets used in training artificial intelligence models and applications. It follows similar moves by other companies to protect their online content. In July, Reddit blocked various search engines, except Google, from indexing its posts and discussions. Google, like Reddit, has a financial agreement with Reddit for data access to train its AI services.

According to sources, in the past year, Microsoft considered restricting access to internet-search data for rival search engine operators; this was most relevant for those who used the data for chatbots and generative AI services.

Meanwhile, the Chinese Wikipedia, with its 1.43 million entries, remains available to search engine crawlers. A survey conducted by the South China Morning Post found that entries from Baidu Baike still appear on both Bing and Google searches. Perhaps the search engines continue to use older cached content.

Such a move is emerging against the background where developers of generative AI around the world are increasingly working with content publishers in a bid to access the highest-quality content for their projects. For instance, relatively recently, OpenAI signed an agreement with Time magazine to access the entire archive, dating back to the very first day of the magazine’s publication over a century ago. A similar partnership was inked with the Financial Times in April.

Baidu’s decision to restrict access to its Baidu Baike content for major search engines highlights the growing importance of data in the AI era. As companies invest heavily in AI development, the value of large, curated datasets has significantly increased. This has led to a shift in how online platforms manage access to their content, with many choosing to limit or monetise access to their data.

As the AI industry continues to evolve, it’s likely that more companies will reassess their data-sharing policies, potentially leading to further changes in how information is indexed and accessed across the internet.

(Photo by Kelli McClintock)

See also: Google advances mobile AI in Pixel 9 smartphones

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is co-located with other leading events including Intelligent Automation Conference, BlockX, Digital Transformation Week, and Cyber Security & Cloud Expo.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

The post Baidu restricts Google and Bing from scraping content for AI training appeared first on AI News.

The tool helps to protect users from certain unsolicited photos – and we’re not just talking about genitalia, but also shirtless selfies and photos of firearms.

When a suspect image is received by a user, it’s blurred to allow the recipient to view it, block it, or report the sender.

Online harassment is difficult to counter entirely but AI is proving a powerful tool in helping to protect users, especially the vulnerable. Shocking research in 2020 found that 75.8 percent of girls between the ages of 12 and 18 have been sent unsolicited nude images.

By open-sourcing its AI tool, Bumble can help to protect more online users.

“It’s our hope that the feature will be adopted by the wider tech community as we work in tandem to make the internet a safer place,” explained Bumble in a blog post.

Human content moderators see the worst of the web day in and day out. Spending your days reviewing abuse, torture, massacres, beheadings, and more is bound to take a serious mental toll. As a result, content moderators often require therapy and is a role associated with one of the highest suicide rates.

Relying solely on AI moderation is problematic. Human moderators, for example, can understand context and tell the difference between content exposing war crimes and that of terrorist propaganda glorifying hate and violence.

AI tools like the one open-sourced by Bumble can help to protect moderators by blurring the content while still harnessing the unique skills of humans.

In its blog post, Bumble explained how it traversed the trade-offs between performance and the ability to serve its user base at scale:

“We implemented (in its latest iteration) an EfficientNetv2-based binary classifier: a convolutional network that has faster training speed and overall better parameters efficiency. It uses a combination of better designed architecture and scaling, with layers like MBConv (that utilizes 1×1 convolutions to wide up the space and depth-wise convolutions for reducing the number of overall parameters) and FusedMBConv (that merges some steps of the vanilla MBConv above for faster execution), to jointly optimize training speed and parameter efficiency.

The model has been trained leveraging our GPU powered data centers in a continuous exercise of dataset, network and hyperparameters (the settings used to speed up or improve the training performance) optimization.”

Bumble says its tool, both offline and online, achieves “world class performance” of over 98 percent accuracy in both upsample and production-like settings.

You can find Bumble’s tool on GitHub here. The tool has been released under the Apache License so anyone can implement it as is for blurring lewd images or can fine-tune it with additional training samples.

(Image Credit: Bumble)

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

The post Bumble open-sources its lewd-spotting AI tool appeared first on AI News.

Ofcom and Cambridge Consultants have teamed up on a report examining the effectiveness of AI-powered online content moderation.

Governments around the world have put increasing amounts of pressure on social networks and communication services to take responsibility for content posted on them. Society itself is becoming more aware of the dangers following live-streamed terror attacks, cyberbullying, political manipulation, and more.

With some platforms having billions of users, manual content moderation of everything posted is not feasible. When illegal content is uploaded, it often requires someone to report it and wait for a human moderator to make a decision (those moderators sometimes require therapy after being exposed to such content).

Ofcom and Cambridge Consultants’ report suggests that AI could help to reduce the psychological impact on human moderators in a few key ways:

• Varying the level and type of harmful content they are exposed to.
• Automatically blurring out parts of the content which the moderator can optionally choose to view if required for a decision.
• Humans can ‘ask’ the AI questions about the content to prepare themselves or know whether it will be particularly difficult for them, perhaps due to past individual experiences.

The slow process of manual content moderation often means harmful content is seen by millions before it’s taken down. While most AI moderation implementations today still require human oversight, some advancements in content detection are helping to speed up content being flagged and removed.

Earlier this month, Facebook-owned Instagram unveiled improvements to an AI-powered moderation system it uses in a bid to prevent troublesome content from ever being posted. While previously restricted to comments, Instagram will now ask users “Are you sure you want to post this?” for any posts it deems may cause distress to others.

As the UK’s telecoms regulator, Ofcom’s report should help to form workable policies rather than generic demands from politicians without a real understanding of how these things work (can anyone remember the calls to ban encryption and/or knowingly create backdoors?)

The report essentially determines that, for the foreseeable future, effective fully automated content moderation is not possible.

Among the chief reasons for fully automated content moderation being problematic is that – while some harmful posts can be identified by analysing it alone – other content requires a full understanding of context. For example, the researchers note how regional and cultural differences in national laws and what’s socially acceptable are difficult for today’s AI moderation solutions to account for but trivial for local human moderators.

Some content is also easier to analyse than others. Photos and pre-recorded videos could be analysed before they’re posted, whereas live-streams pose a particular difficulty because what appears to be an innocent scene could become harmful very quickly.

“Human moderators will continue to be required to review highly contextual, nuanced content,’ says Cambridge Consultants’ report. “However, AI-based content moderation systems can reduce the need for human moderation and reduce the impact on them of viewing harmful content.”

You can find a copy of the full report here (PDF)

Interested in hearing industry leaders discuss subjects like this and their use cases? Attend the co-located AI & Big Data Expo events with upcoming shows in Silicon Valley, London, and Amsterdam to learn more. Co-located with the IoT Tech Expo, Blockchain Expo, and Cyber Security & Cloud Expo.

The post Ofcom: AI is not ready to effectively moderate online content ‘for the foreseeable future’ appeared first on AI News.