Hard Problems in Terms of Service Enforcement

Accepted Session
Short Form
Scheduled: Wednesday, June 22, 2016 from 4:45 – 5:30pm in B202/203


When you run an online service, you always hope you won't have to deal with abuse. But it's inevitable, and many situations aren't clear-cut as you might wish. Some examples of abuse are obvious, but this talk explores the grey areas and messy questions: what content should you consider a violation of your Terms of Service, and how do you handle it when it's reported to you?


What do you do when a whistleblower posts something on your platform that exposes a major company’s unsafe, unethical, or illegal business practices — and contains something the company claims is a trade secret and needs to be taken down immediately?

What do you do when a teenager posts gossip about a classmate’s love life, and that classmate’s parent demands your service remove the post or else they’ll sue you?

What do you do when you keep getting repeated Terms of Service violation reports about an account, but you’re positive that the violations are being manufactured by a former friend to try to get someone’s account closed?

What do you do when someone has a restraining order against an ex, and they think that ex has made an alternate account to leave comments for them — but you can’t prove it?

While there aren’t any simple answers, if you’ve built a platform for people to people to post content of any sort online, these are questions you should think about before you open your doors to the public. Any online platform with more than two users will inevitably contain content that violates that platform’s Terms of Service. No matter how many automated anti-abuse tools your product includes, you’ll soon find yourself having to look at a situation and determine who’s violated your Terms of Service and what should be done with the violation. When human judgement is fallible and bright-line tests don’t cover everything, how do you decide actions to take, and whom do you empower to make those decisions?

Online interpersonal abuse is a serious problem, but it isn’t the only sort of complaint you’ll get. Using real-world case studies from fifteen years with LiveJournal.com and Dreamwidth.org, we’ll discuss the sort of things people report as a Terms of Service violation, explore many of the grey areas you’ll encounter, and talk about the ways a service can — and should — respond to Terms of Service complaints. If you’re planning to start an online service or work for one and want to advocate for Terms of Service enforcement improvements, this is the talk for you.

(Though it focuses on scenarios other than mobbing, doxxing, SWATting, etc, this talk will contain examples of abusive, harassing, or just plain unpleasant content.)


terms of service, platform management, community management

Speaking experience

I've spoken at Open Source Bridge for the past few years, and also at OSCON, Linux Conf Australia, YAPC::NA, and O'Reilly's Web 2.0 Expo, among others. This talk is new this year.


Leave a private comment to organizers about this proposal