A single model has a single failure mode. An ensemble of models can compensate for individual weaknesses, improve accuracy, and provide built-in redundancy. But ensembles add complexity, cost, and latency that must be justified by measurable improvement.

Voting Ensemble

Multiple models process the same input independently, and the final output is determined by majority vote (for classification) or averaging (for scores and rankings).

When to use it - When accuracy on critical decisions justifies the cost of multiple inference calls. Fraud detection, medical diagnosis assistance, and safety-critical classifications benefit from voting ensembles because the cost of a wrong answer far exceeds the cost of extra compute.

Implementation - Run the same input through 3-5 models (different model families or different fine-tuned versions of the same base). For classification tasks, take the majority vote. For extraction tasks, take the answer that appears most frequently. Flag cases where models disagree strongly - these are the cases most likely to be genuinely ambiguous.

Disagreement as signal - Model disagreement is itself valuable information. High disagreement indicates an input that is inherently ambiguous or near a decision boundary, and may warrant human review regardless of the majority vote. Track disagreement rates by input type to identify where the ensemble struggles.

Cascade Ensemble

A chain of models where a fast, cheap model handles easy cases and progressively more capable (and expensive) models handle harder ones. This optimizes cost while maintaining accuracy.

When to use it - When input difficulty varies widely and most inputs are straightforward. Support ticket classification where 70% of tickets are clearly categorized by a small model, 25% need a medium model, and 5% need the most capable model is a classic cascade use case.

Implementation - The first model in the cascade processes the input and produces an output with a confidence score. If confidence exceeds the threshold, the output is used. If not, the input passes to the next model. Each tier has its own confidence threshold calibrated independently.

Cost optimization - The cascade’s value comes from the distribution of difficulty. If 80% of inputs resolve at the cheapest tier, the average cost per inference drops significantly even though the most capable model is expensive. Track the percentage of inputs resolved at each tier and optimize thresholds to minimize cost while meeting accuracy targets.

Specialization Ensemble

Different models handle different types of inputs, each optimized for their specialty. A router directs each input to the appropriate specialist.

When to use it - When input types have fundamentally different characteristics that benefit from specialized handling. A document processing system might use one model optimized for invoices, another for contracts, and another for correspondence, each fine-tuned on domain-specific training data.

Implementation - A lightweight classifier routes incoming inputs to the appropriate specialist model. The classifier itself can be a simple model since it only needs to identify the input type, not produce the final output. Each specialist model is independently tuned and evaluated on its specific input type.

Routing failures - The router is a single point of failure. If it misclassifies an input type, the wrong specialist processes it, likely producing poor results. Include a catch-all specialist that handles unrecognized input types and routes them to human review rather than guessing.

Cross-Validation Ensemble

One model produces the output, and a second model validates it. This is particularly useful for generative tasks where the output should satisfy specific constraints.

When to use it - When the output has verifiable properties. Generated code can be validated by a different model checking for bugs. Extracted data can be validated by a model checking for consistency. Summaries can be validated by a model checking for factual accuracy against the source.

Implementation - The generator model produces output. The validator model receives both the original input and the generated output, and checks for specific quality criteria. If validation fails, the output is regenerated with the validation feedback included in the prompt, or routed to human review.

Practical Considerations

Latency impact - Parallel ensembles (voting) add the latency of the slowest model. Sequential ensembles (cascade, cross-validation) add the latency of each model in the chain. For user-facing applications, latency budgets may constrain ensemble design.

Cost modeling - Model the total cost per input for each ensemble strategy against the single-model baseline. Include the cost of disagreement resolution and human review for edge cases. The ensemble must deliver measurable accuracy improvement that justifies the cost increase.

Monitoring complexity - Each model in the ensemble needs independent monitoring for drift, accuracy degradation, and latency. The ensemble as a whole needs monitoring for agreement rates, tier utilization (cascades), and routing accuracy (specialization). Build ensemble-level dashboards from day one.