- Problem
- Product teams tracked competitor Trust & Safety policies by hand, monthly at best.
- My role
- Senior PM. I scoped, designed, and built it end to end.
- What I built
- A monitor that checks 25+ policies every 6 hours and emails AI summaries of real changes.
- Results
- Alerts in under 6 hours on average, over 95% precision, 99.9% uptime, 10+ PM hours saved a month.
The problem
In social commerce and live streaming, Trust & Safety policy is strategy. A new TikTok guideline or Meta commerce rule can show where the market is going. It can open an advantage or expose a risk.
Product teams tracked these changes by hand. They checked maybe once a month and missed most meaningful changes. They heard about updates weeks late, with no history to compare against. PM hours went to routine checking instead of analysis.
I had spent years as a Trust & Safety PM. This was work a machine should do.
What success looked like
One source of truth that runs with no upkeep. Three bars had to hold:
- Fast: hear about a change within hours, not weeks
- Precise: only real changes. One noisy alert and people stop reading.
- Trusted: the system reports its own health, so silence means "nothing changed," not "something broke"
Key decisions
Precision over coverage
The hard part was telling real policy edits from cosmetic site changes. Ads, timestamps, and session IDs made pages look new on every run. YouTube looked changed daily while the text was identical.
I chose to filter aggressively. The pipeline strips 20+ kinds of dynamic content and compares both page structure and plain text. Missing a tiny edit costs less than a false alarm, because false alarms train people to ignore the tool.
Summaries about impact, not diffs
Raw diffs are useless to a busy PM. Each change goes to Google Gemini with one question: what changed, and why does it matter for product strategy? Alerts arrive as plain text email, which reads cleanly on a phone, where executives often check them.
Two fetchers for bot-protected sites
TikTok, Meta, and Whatnot block ordinary scrapers. Plain HTTP requests handle simple pages. Playwright browser automation handles protected ones. Every site gets covered, and simple ones stay fast.
Zero maintenance as a requirement
It runs on GitHub Actions and recovers from errors on its own. A run log and health checks surface any failure, so problems get fixed before anyone notices stale data.
Results
- 25+ policies tracked across TikTok, YouTube, Meta, Whatnot, and Twitch
- Under 6 hours on average from a policy change to the alert, down from weeks
- Over 95% precision telling real changes from cosmetic ones
- 99.9% uptime with no silent failures
- 10+ hours a month saved per PM, by estimate
Takeaways
- Internal tools need product rigor. Automating the checks was the easy part. Alert volume and trust decided whether anyone used it.
- For alerting, precision beats recall. A tool that cries wolf gets muted.
- Reliability is a feature. If people rely on it, uptime belongs on the roadmap.
What's next
More platforms (Discord, Snapchat, LinkedIn), policies outside the US, and trend analysis across platforms to flag competitive openings.
The Policy Watcher is live at ts-policy-watcher.vercel.app.