Design Content Moderation at Scale — System Design Practice | AmanAI Lab

Sheet›Design Content Moderation at Scale

MetaGoogleAmazonOpenAI

45:00

Hard

The Problem

Problem

Design a content moderation system for a social media platform with 500 million posts per day that must detect and remove harmful content (hate speech, CSAM, violence, spam, misinformation) while minimising false positives on legitimate content.

What you'll be assessed on

Multi-stage moderation pipeline, human review queues, adversarial adaptation, the FP/FN trade-off for different content categories, and appeals handling.

Scale & Constraints

▸500M posts/day across text, images, and video
▸Harmful content removed within 1 hour of posting (regulatory requirement)
▸CSAM removed within 5 minutes (legal requirement)
▸False positive rate ≤ 0.01% for mainstream content
▸System must adapt to new harmful content patterns weekly
▸Multi-language support (100+ languages)

Must Cover0/9

Hints (if stuck)

💡 Hash-based matching (PhotoDNA) is instant and 100% recall for known content — first stage.

💡 Active learning: send borderline predictions (0.4–0.6 confidence) to human review for highest labelling ROI.

💡 Context matters: "I killed it on the presentation" is different from a literal threat — use conversation context.

💡 FN (missed harmful content) is far more costly than FP (false removal) for CSAM — adjust thresholds accordingly.

0 words · auto-saved

Back to sheet

Mock interviewTake quizFlashcards