Gemini API Content Moderation and Policy Enforcement

Built-in safety filters block clearly harmful content, but every business has its own content policies. ShopMax India needs to detect fake reviews, misleading product claims, competitor brand names in listings, and pricing that violates platform rules. Custom content moderation with Gemini goes beyond safety filters to enforce your specific business rules.

You define a policy document and pass content through a moderation agent that checks against each rule. The response includes which rules were violated, a severity score, and a recommended action. This is more accurate and explainable than keyword lists or regex patterns.

The below example shows how ShopMax India moderates product listings and customer reviews using Gemini.

Moderating a product listing and a review,

It gives the following output,

Listing moderation:
{
  "approved": false,
  "violations": [
    {"rule": 2, "reason": "Mentions competitor brand iPhone", "severity": "high"},
    {"rule": 4, "reason": "Unverifiable claim: best smartphone, last forever", "severity": "medium"},
    {"rule": 5, "reason": "Phone number 9876543210 in description", "severity": "high"}
  ],
  "action": "reject",
  "notes": "3 policy violations. Remove competitor mention, personal contact, and unverifiable claims."
}

Review moderation:
{
  "approved": false,
  "violations": [{"rule": 1, "reason": "Indicates free product received for review", "severity": "high"}],
  "action": "reject",
  "notes": "Incentivised review detected. Not eligible for publication."
}

For production at ShopMax India, run moderation asynchronously on all new listings and reviews before they go live. Flag medium severity violations for human review and auto-reject high severity violations. Log all moderation decisions with the violation reasons to Firestore for dispute resolution and policy refinement.