Document classification assigns one or more labels to a document based on its content. It is among the most common AI tasks in enterprise applications: routing incoming correspondence, categorizing support tickets, tagging content for search, and classifying documents for compliance.

Classification Approaches

Zero-shot classification - Use an LLM to classify documents without task-specific training data. Provide the category definitions in the prompt and ask the model to assign the most appropriate category. This works immediately with no data collection and handles new categories by updating the prompt. Best for low-volume use cases or rapid prototyping.

Few-shot classification - Include 3-5 examples of each category in the prompt. This improves accuracy significantly over zero-shot, especially for nuanced categories where the boundary between classes is ambiguous. The examples teach the model your specific interpretation of each category.

Fine-tuned classification - Train a model specifically for your classification task using labeled training data. This produces the highest accuracy and lowest inference cost but requires upfront data labeling effort and ongoing maintenance as categories evolve. Best for high-volume, stable classification tasks.

Multi-Label Classification

Documents often belong to multiple categories simultaneously. A customer complaint about billing might also be a feature request and a churn risk signal.

Independent scoring - Score each label independently rather than forcing a single choice. Return a confidence score for each possible label and apply a threshold to determine which labels are assigned. This allows a document to receive zero, one, or many labels.

Hierarchical classification - For taxonomies with parent-child relationships (Industry > Finance > Banking), classify at the leaf level and inherit parent labels. This ensures consistent hierarchy and enables search at any level of specificity.

Primary vs. secondary labels - When consumers need a single primary classification for routing purposes but want additional labels for analytics, separate primary classification (single label, highest confidence) from secondary tagging (multiple labels above threshold).

Confidence Handling

Not all classifications are equally certain. Handling low-confidence classifications correctly is the difference between a useful system and a frustrating one.

Threshold calibration - Set confidence thresholds based on the cost of misclassification. For high-stakes classifications (compliance, security), use a high threshold and route uncertain items to human review. For low-stakes classifications (content tagging), use a lower threshold and accept some errors.

Rejection option - Include an explicit “uncertain” or “other” classification for documents that do not fit any defined category well. This is better than forcing every document into an existing category with low confidence.

Confidence calibration - Model confidence scores are often poorly calibrated (a 90% confidence may only be correct 70% of the time). Calibrate confidence scores against actual accuracy using historical data. This ensures thresholds produce the expected error rates.

Taxonomy Management

The classification taxonomy changes over time as business needs evolve. Managing these changes without breaking the system is an ongoing challenge.

Category lifecycle - New categories start in a “provisional” state where all assignments are reviewed. Once accuracy reaches the target threshold, the category moves to “production” with standard review. Deprecated categories are phased out by remapping to successor categories.

Taxonomy versioning - Version the taxonomy alongside the classification model. When the taxonomy changes, retrain or re-prompt the model, evaluate on the new category set, and track metrics per taxonomy version. This enables rollback if a taxonomy change degrades overall accuracy.

Feedback integration - Users who consume classifications should have a mechanism to flag incorrect ones. Feed these corrections back into training data and use them to identify categories that need clearer definitions or additional examples.