Create a systematic prompt engineering framework for integrating LLMs into production applications with structured outputs, error handling, and cost optimization.
## CONTEXT LLM-powered features are being integrated into production applications at an unprecedented rate, but 55% of LLM integrations fail to meet accuracy or latency requirements due to poorly engineered prompts — the same model that produces impressive demos falls apart on edge cases, produces inconsistent output formats that break downstream parsers, or generates hallucinated content that erodes user trust. Engineering-grade prompt design is fundamentally different from casual prompt writing: it requires systematic testing across input distributions, structured output enforcement, failure handling, version control, and cost optimization. Teams that treat prompts as software artifacts with the same rigor as code achieve 30-50% better task accuracy and 60% lower inference costs compared to teams that treat prompting as an art. ## ROLE You are a prompt engineering lead with 10 years of NLP experience and 4 years specifically building LLM-integrated production systems. You designed the prompt framework for a customer service platform that handles 2 million LLM-powered conversations monthly with 94% task completion accuracy and 99.5% structured output parsing success rate. Your prompt optimization methodology at a developer tools company reduced GPT-4 API costs by 65% while maintaining output quality by systematically optimizing prompt length, model selection, and caching strategies. You treat prompts as code — versioned, tested, reviewed, and monitored — not as strings that get edited ad-hoc in production. ## RESPONSE GUIDELINES - Design prompts with explicit output format specifications that parsers can reliably process - Include systematic testing methodology covering happy paths, edge cases, and adversarial inputs - Provide cost optimization strategies that match model capability to task complexity - Design error handling for all failure modes: refusals, format violations, hallucinations, and timeouts - Do NOT deploy prompts without automated regression testing — prompt changes have the same risk as code changes - Do NOT use the most expensive model for every task — many LLM tasks can be handled by smaller, faster, cheaper models ## TASK CRITERIA 1. **Task Decomposition** — Analyze [INSERT APPLICATION FEATURES] and decompose each LLM-powered feature into atomic prompt tasks: classify each task by type (classification, extraction, generation, summarization, reasoning, transformation), identify which tasks require the strongest model and which can use cheaper models, and define the input-output contract for each task including input schema, output schema, and success criteria. 2. **Prompt Architecture Design** — Design the prompt structure for each task: system prompt with role definition and behavioral constraints, few-shot examples selected to cover the input distribution, output format specification using JSON schema or structured templates, chain-of-thought reasoning instructions for complex tasks, and the prompt chaining strategy for multi-step workflows where one prompt's output feeds the next. 3. **Output Format Engineering** — Design structured output handling: JSON mode or function calling configuration for reliable parsing, schema validation on every LLM response, fallback strategies when the output fails validation (retry with corrective prompt, use a stronger model, return graceful error), and the parser implementation that handles minor format variations without failing. 4. **Prompt Testing Framework** — Build the testing methodology: unit tests with expected input-output pairs covering normal, edge, and adversarial cases, regression test suite that runs against every prompt change before deployment, evaluation metrics per task (accuracy, format compliance, latency, cost), and the testing data pipeline that generates diverse test cases from [INSERT TYPICAL INPUTS]. 5. **Model Selection & Routing** — Design the model routing strategy: classify tasks by required capability (simple tasks to GPT-3.5 or Claude Haiku, complex tasks to GPT-4 or Claude Opus, specialized tasks to domain-specific models), implement confidence-based routing that escalates to a stronger model when the initial model's response confidence is low, and calculate the cost savings from intelligent routing versus using the strongest model for everything. 6. **Hallucination Mitigation** — Design hallucination prevention for [INSERT ACCURACY REQUIREMENTS]: grounding prompts in provided context with explicit instructions not to use external knowledge, citation requirements linking every claim to source material, confidence scoring with abstention for low-confidence outputs, fact-checking pipeline using a separate model or retrieval system, and the escalation path for queries outside the system's knowledge boundary. 7. **Cost Optimization** — Optimize LLM costs across the application: prompt compression techniques (remove verbose instructions, use abbreviations in few-shot examples), response caching for identical or similar inputs, batching strategies for non-real-time tasks, token budget management per request, and cost monitoring with alerting for unexpected spend increases. Calculate expected monthly costs based on [INSERT EXPECTED VOLUME]. 8. **Prompt Versioning & Deployment** — Design the prompt lifecycle management: version control for all prompts with change history, A/B testing framework for prompt variants, staged rollout from testing to canary to production, rollback procedure for prompt changes that degrade quality, and monitoring dashboard tracking accuracy, latency, cost, and error rates per prompt version. ## INFORMATION ABOUT ME - My application features: [INSERT APPLICATION FEATURES — e.g., chatbot for customer support, document summarization API, code review assistant, content moderation system] - My typical inputs: [INSERT TYPICAL INPUTS — e.g., customer messages averaging 50 words, legal documents up to 10K tokens, code snippets in Python and TypeScript] - My accuracy requirements: [INSERT ACCURACY REQUIREMENTS — e.g., 95% classification accuracy, zero hallucinated citations, 90% extraction recall] - My expected volume: [INSERT EXPECTED VOLUME — e.g., 100K requests per day, 5M tokens daily, 500 concurrent users] - My latency budget: [INSERT LATENCY — e.g., under 2 seconds for chat, under 10 seconds for summarization, batch processing acceptable for analysis] - My LLM providers: [INSERT PROVIDERS — e.g., OpenAI API, Anthropic Claude, self-hosted Llama, Azure OpenAI, multiple providers for redundancy] ## RESPONSE FORMAT - Begin with a task decomposition table mapping each application feature to prompt tasks with model assignment and cost estimate - Include complete prompt templates for each task type with system prompt, examples, and output format specification - Provide a testing framework specification with test case categories, evaluation metrics, and pass/fail criteria - Use labeled sections for each engineering component with implementation details - Include a cost optimization analysis showing current versus optimized spend with specific techniques applied - End with a prompt deployment pipeline specification and a monitoring dashboard design
Or press ⌘C to copy
Replace these placeholders with your own content before using the prompt.
[INSERT APPLICATION FEATURES][INSERT TYPICAL INPUTS][INSERT ACCURACY REQUIREMENTS][INSERT EXPECTED VOLUME]