Own model or an LLM with a prompt for content moderation
There is a simple way to moderate: paste your rules into the prompt of a large model and send it the content. Sometimes that is enough. Below, when it is and when it is worth considering your own model.
- An LLM with a prompt is good to start with and at small scale.
- An own model gains as volume grows and consistency and cost matter.
- You will choose best after comparing both on your own data.
Two approaches
In the first you send each piece of content to a large language model together with a description of your rules and ask for a verdict. In the second the model is tuned on your data and rules, so the rules live in the model, not in the prompt.
When an LLM with a prompt is enough
At low volume, when the rules change every few days, when you have no labelled examples and when you are building a prototype. In those cases an own model is needless cost and time.
When to consider an own model
When you count content in hundreds of thousands or millions, when you want a predictable cost, when you have a history of moderator decisions, when content cannot leave your environment, or when the same content must get the same decision every time.
What to watch out for with a prompt
Cost depends on the number of tokens, and a long description of rules is added to every request.
The answer is free text you have to turn into a decision, and the same content can get different scores on repeated calls.
The moderated content can contain instructions aimed at the model, for example "ignore the previous rules". You need to defend against that.
Don't assume an own model always wins
Tuning does not always give a big jump in quality over a good general model. It depends on the task, the language and the amount of data. That is why in a pilot we compare both approaches on your sample and only then recommend one. If a prompt is enough, we will say so.
Four questions to start with
Do you moderate hundreds of thousands of items a month? Do you have a history of moderator decisions? Must the content stay inside your environment? Do you need a consistent reason for every decision? The more answers are yes, the more sense an own model makes.