Moderation

Astroturfing is a coordinated campaign disguised as a spontaneous grassroots movement, where paid or organized actors pose as ordinary users to push a political, commercial, or ideological message. It typically relies on networks of sock puppet accounts to manufacture the appearance of broad public support.

Bot detection is the identification of automated accounts and scripted traffic through a combination of behavioral analysis, device fingerprinting, network signals, and challenge-response checks. It underpins abuse prevention across spam, fraud, scraping, and inauthentic engagement.

Coordinated inauthentic behavior is a term for networks of fake or compromised accounts that work together to mislead users about their identity and purpose, often to manipulate political discourse, amplify narratives, or run influence operations. Detection focuses on cross-account signals such as shared infrastructure, posting patterns, and content overlap.

CSAM stands for Child Sexual Abuse Material — any visual depiction of sexually explicit conduct involving a minor. CSAM is illegal in virtually every jurisdiction, and US-based platforms are legally required to report known instances to the National Center for Missing & Exploited Children (NCMEC).

Grooming detection is the conversation-level identification of patterns used by adults to build trust with minors for the purpose of sexual exploitation. It relies on analyzing multi-turn dialogue — including isolation tactics, boundary testing, and escalation toward sexual content — rather than judging single messages in isolation.

Hash matching is a detection technique that compares the cryptographic or perceptual fingerprint of a piece of content against a database of known-harmful material, enabling instant identification without re-analyzing the media. It is the backbone of industry efforts to remove CSAM and terrorist content at scale.

An imposter scam is a fraud in which the attacker pretends to be someone the victim trusts — a government agency, a family member, a business, or a tech support agent — in order to extract money or personal information. Imposter scams are consistently the top category of fraud reported to the US Federal Trade Commission, with billions of dollars in reported losses each year.

Non-consensual intimate imagery refers to sexually explicit photos or videos shared without the subject's consent, often called "revenge porn". It is now criminalized in many jurisdictions and is typically handled through dedicated victim-reporting workflows, hash-sharing programs, and expedited takedown procedures.

PhotoDNA is a hash-matching technology developed by Microsoft that creates a robust digital signature of an image, allowing platforms to detect known child sexual abuse material even after cropping, resizing, or minor edits. It is widely deployed across major platforms and reported into NCMEC's CyberTipline.

A pig butchering scam is a long-con fraud in which a scammer builds a romantic or friendly relationship with a victim over weeks or months, then lures them into a fake cryptocurrency investment platform and drains their savings. The name comes from the Chinese "sha zhu pan" — fattening the pig before the slaughter.

A romance scam is a fraud in which the attacker feigns a romantic relationship with the victim to win their trust and then solicit money, gift cards, or cryptocurrency. Romance scams are frequently the entry point for pig butchering investment fraud and financial sextortion, and are among the most costly categories of consumer fraud tracked by the FBI and FTC.

Self-harm detection is the identification of user-generated content that expresses suicidal ideation, self-injury, or eating disorder behaviors. Best practice is to route such content to crisis support resources and trained human reviewers rather than issuing blunt removals, which can compound harm for vulnerable users.

Sextortion is a form of online blackmail in which an attacker threatens to share sexual images or videos of a victim unless they pay money or provide more content. The FBI has warned of a sharp rise in financial sextortion targeting teenage boys, with tens of thousands of reports filed to the National Center for Missing & Exploited Children each year.

Shadow banning is the practice of silently reducing the visibility of a user's posts — hiding them from feeds, search, or replies — without notifying the user that any action has been taken. Platforms use it to contain spammers and trolls without giving them a signal they can adapt to.

SHAFT is a content moderation and advertising compliance acronym for Sex, Hate, Alcohol, Firearms, and Tobacco — five categories of regulated or sensitive material that ad networks and platforms restrict, age-gate, or prohibit outright.

A sock puppet is a fake online identity created to deceive other users, usually operated by someone who already has a primary account. Sock puppets are the basic building block of coordinated inauthentic behavior, astroturfing, and review manipulation.

Trust & Safety is the discipline within an online platform responsible for protecting users from harm — covering content moderation, fraud prevention, policy enforcement, and incident response. Content moderation is one pillar of a broader Trust & Safety function that also handles abuse, scams, and regulatory compliance.