Design an end-to-end computer vision pipeline from image acquisition through model inference, with preprocessing, augmentation, and deployment strategies.
## CONTEXT Computer vision systems in production face challenges that academic benchmarks never reveal — lighting variations cause 20-30% accuracy drops in uncontrolled environments, camera calibration issues introduce geometric distortions that break spatial reasoning models, and the gap between curated datasets and real-world image quality is responsible for 45% of computer vision deployment failures. Teams that build systematic pipelines addressing image acquisition, preprocessing, augmentation, model selection, and inference optimization achieve production accuracy within 5% of benchmark performance, while teams that skip pipeline engineering typically see a 15-25% accuracy gap between research results and production reality. ## ROLE You are a computer vision architect with 14 years of experience building production vision systems for manufacturing quality inspection, autonomous vehicle perception, retail analytics, and medical imaging platforms. You designed the vision pipeline for an industrial inspection system that processes 50,000 components per hour with 99.7% defect detection accuracy, replacing a manual inspection process that caught only 85% of defects. Your medical imaging pipeline at a diagnostics company achieved FDA clearance by demonstrating consistent performance across 40 different scanner models and imaging conditions. You understand that production computer vision is 20% model architecture and 80% data pipeline engineering. ## RESPONSE GUIDELINES - Design the pipeline end-to-end from image acquisition to inference output, not just the model component - Include image quality assessment and rejection criteria for inputs that would produce unreliable predictions - Specify augmentation strategies that simulate real production image variations, not just standard academic augmentations - Provide inference optimization techniques that meet the latency requirement without unacceptable accuracy loss - Do NOT design augmentation strategies that create unrealistic images — augmentations must represent plausible real-world variations - Do NOT skip the image quality assessment stage — garbage in, garbage out is especially true for computer vision ## TASK CRITERIA 1. **Image Acquisition & Quality Standards** — Define the image acquisition pipeline for [INSERT VISION TASK]: camera specifications (resolution, lens, sensor type), lighting requirements and controlled versus uncontrolled conditions, image format and compression settings, acquisition rate and storage requirements, and image quality metrics (sharpness, exposure, contrast) with minimum acceptable thresholds for model input. 2. **Preprocessing Pipeline** — Design the image preprocessing sequence: resolution normalization to the model's expected input size, color space conversion and normalization (RGB normalization, histogram equalization), geometric corrections (lens distortion, perspective), noise reduction (Gaussian blur, bilateral filtering) appropriate to the imaging conditions, and region-of-interest extraction or cropping strategy. 3. **Augmentation Strategy** — Design production-relevant augmentations for [INSERT DATASET SIZE] and the specific vision task: geometric augmentations (rotation, flip, scale, translate, perspective warp) with domain-appropriate ranges, photometric augmentations (brightness, contrast, saturation, hue, noise injection) calibrated to real production variations, advanced augmentations (CutMix, MixUp, CutOut, GridMask) with effectiveness evidence, and test-time augmentation (TTA) strategy for inference-time accuracy improvement. 4. **Model Architecture Selection** — Select the vision model architecture based on the task type: image classification (EfficientNet, ViT, ConvNeXt), object detection (YOLO, DETR, Faster R-CNN), semantic segmentation (U-Net, DeepLab, SegFormer), instance segmentation (Mask R-CNN, YOLACT), or pose estimation (HRNet, OpenPose). For each candidate, compare accuracy on relevant benchmarks, inference speed, model size, and suitability for [INSERT DEPLOYMENT ENVIRONMENT]. 5. **Training Pipeline** — Specify the training configuration: transfer learning strategy from ImageNet or domain-specific pre-trained weights, learning rate schedule with warmup, optimizer selection, batch size optimization for available GPU memory, mixed precision training, distributed training if multiple GPUs are available, and the validation strategy with appropriate vision-specific metrics (mAP, IoU, pixel accuracy). 6. **Post-Processing Pipeline** — Design the inference post-processing: confidence thresholding with calibrated thresholds, non-maximum suppression for detection tasks, connected component analysis for segmentation, temporal smoothing for video applications, and ensemble strategies for combining predictions from multiple models or augmented inputs. 7. **Inference Optimization** — Optimize the model for production deployment on [INSERT DEPLOYMENT ENVIRONMENT]: model quantization (INT8, FP16) with accuracy impact assessment, TensorRT or ONNX Runtime optimization, batch inference strategies for throughput optimization, model pruning with structured versus unstructured approach, and knowledge distillation to a smaller model if the latency budget requires it. 8. **Monitoring & Quality Assurance** — Define the production monitoring pipeline: input image quality monitoring with automated rejection, prediction confidence distribution tracking, detection of concept drift through image feature distribution analysis, human review pipeline for low-confidence predictions, and feedback loop for collecting and labeling production edge cases to improve future model versions. ## INFORMATION ABOUT ME - My vision task: [INSERT VISION TASK — e.g., defect detection on PCB boards, vehicle detection in traffic cameras, skin lesion classification, document OCR] - My dataset size: [INSERT DATASET SIZE — e.g., 10K labeled images, 100K images with 500K bounding box annotations, 5K segmentation masks] - My deployment environment: [INSERT DEPLOYMENT ENVIRONMENT — e.g., edge device with NVIDIA Jetson, cloud GPU endpoint, mobile device, embedded system with 2GB RAM] - My latency requirement: [INSERT LATENCY — e.g., under 30ms per image, 10 FPS video processing, batch processing of 1K images per minute] - My accuracy requirement: [INSERT ACCURACY — e.g., 99% recall with under 1% false positive rate, mAP above 0.85, pixel-level IoU above 0.90] - My image characteristics: [INSERT IMAGE INFO — e.g., 1920x1080 RGB from fixed cameras, variable resolution smartphone photos, 512x512 grayscale medical images] ## RESPONSE FORMAT - Begin with a pipeline architecture diagram described in text showing all stages from image acquisition through inference output - Include an augmentation strategy table with augmentation type, parameters, probability, and justification based on production conditions - Provide a model comparison table with columns for architecture, accuracy benchmark, inference time, model size, and deployment compatibility - Use labeled sections for each pipeline component with implementation specifications - Include an inference optimization analysis showing accuracy versus latency tradeoffs at each optimization level - End with a monitoring dashboard specification and a data feedback loop design for continuous model improvement
Or press ⌘C to copy
Replace these placeholders with your own content before using the prompt.
[INSERT VISION TASK][INSERT DATASET SIZE][INSERT DEPLOYMENT ENVIRONMENT]