The rise of large language models has fundamentally changed the academic integrity landscape. Traditional plagiarism detection - matching text against a corpus of known sources - cannot detect AI-generated original text. Institutions need detection systems that go beyond text matching to include stylometric analysis, semantic similarity detection, and AI-generated content identification.

The Problem

Traditional plagiarism tools like Turnitin rely primarily on string matching against databases of published works, web content, and previously submitted papers. This approach has two blind spots: paraphrased plagiarism (rewriting a source in different words) and AI-generated content (original text produced by an LLM that matches no existing source). With AI writing tools freely available, the proportion of submissions containing AI-generated content has increased significantly, and institutions lack reliable detection mechanisms.

AI Approach

Semantic similarity detection - Rather than matching exact strings, SageMaker embedding models convert text passages into semantic vectors. Passages with similar meaning but different wording are detected by measuring cosine similarity between embeddings. This catches paraphrased plagiarism that string-matching tools miss. The reference corpus is indexed in Amazon OpenSearch with vector search capability.

Stylometric profiling - Each student develops a writing fingerprint over multiple submissions: vocabulary diversity, sentence structure patterns, punctuation habits, and argumentation style. A deviation model flags submissions that differ significantly from the student’s established profile. This approach detects both human-written plagiarism (someone else wrote it) and AI-generated content (the writing style differs from the student’s baseline).

AI-generated content detection - Dedicated classifiers trained to distinguish human-written from AI-generated text analyze statistical properties: perplexity distribution, token probability patterns, and burstiness (humans write with more variable sentence complexity than LLMs). These detectors are not perfectly reliable - accuracy ranges from 70% to 90% depending on the model and the sophistication of the AI-generated text - and should inform investigation rather than determine outcomes.

Source attribution - Bedrock analyzes flagged passages to identify potential source material, generating explanations of why a passage was flagged and what evidence supports the detection. This gives instructors the context needed to make informed decisions.

Architecture

Student submissions flow from the LMS to S3. A Step Functions workflow orchestrates the detection pipeline: text extraction, embedding generation, similarity search against the reference corpus in OpenSearch, stylometric comparison against the student’s profile, and AI-generation probability scoring. Results are aggregated into an integrity report stored in DynamoDB and surfaced to instructors through the LMS integration.

Key Considerations

False positive management - No detection system is infallible. AI-generated content detectors in particular have significant false positive rates, especially for non-native English speakers whose writing may statistically resemble AI output. Detection results must be treated as evidence for investigation, not as proof of misconduct.

Due process - Automated detection should trigger a human review process, not automatic penalties. Students must have the opportunity to explain flagged submissions before any academic consequences.

Baseline building - Stylometric profiling requires multiple samples of a student’s authentic writing. In-class writing samples collected under supervised conditions provide the most reliable baseline.

Arms race dynamics - Detection and evasion techniques evolve in parallel. Systems must be updated regularly as new AI writing tools and obfuscation techniques emerge.

Next Steps

Deploy semantic similarity search alongside existing string-matching tools to catch paraphrased plagiarism immediately. Collect supervised writing baselines for incoming students. Introduce AI-generation detection as an investigative aid with clear institutional policies about how flagged results are handled and communicated.