Vendor Templates
Pre-built vendor evaluation templates help you quickly compare LLM providers. Choose from RFP templates, scorecard templates, or comparison matrices. Customize and share with vendors or your team.
Available Templates
1. LLM Provider RFP (Request for Proposal)
Comprehensive questionnaire for vendors to complete.
Sections:
- Company background (financial stability, customer base)
- Security & compliance (certifications, audit practices)
- Product capabilities (models, features, performance)
- Service & support (SLA, response times, training)
- Pricing (cost structure, volume discounts, hidden fees)
- Data privacy (data usage, retention, residency options)
- Contracts & terms (lock-in, exit clauses, dispute resolution)
Use when: Evaluating vendors for large contracts or multi-year commitments
Time to complete: 1-2 hours for vendor (detailed questions)
Output: Comprehensive information for deep evaluation
2. Quick Vendor Scorecard
Fast evaluation template (20 questions, 30 min).
Covers:
- Model quality on your use cases (quick test)
- Cost (API pricing structure)
- Security baseline (certifications)
- Support availability (SLA options)
- Data privacy (default practices)
Use when: Initial vendor screening; you have 3-5 candidates to evaluate
Time to complete: 30 minutes
Output: Quick ranking of finalists
3. RFP Template for Custom Evaluation
Generic RFP you customize for your org’s needs.
Customization options:
- Add/remove sections based on what matters to you
- Adjust technical questions for your domain
- Include company-specific requirements (data residency, certifications, etc.)
Sample sections:
- Executive summary (vendor background)
- Technical requirements (model specs, API features)
- Security & compliance requirements
- Pricing & contract terms
- Support & SLA requirements
- References (who else uses them?)
4. Feature Comparison Matrix
Side-by-side comparison of technical features.
Rows: Your feature requirements Columns: Vendor candidates
Example for code generation:
| Feature | Requirement | GPT-4 | Claude | Gemini |
|---|---|---|---|---|
| Multi-language support | Must have | Yes | Yes | Yes |
| Function calling | Nice-to-have | Yes | Yes | No |
| Fine-tuning | Nice-to-have | Yes | No | Yes |
| Vision (code in images) | Nice-to-have | Yes | No | No |
| Custom system prompt | Must have | Yes | Yes | Yes |
Use when: Evaluating vendors on technical features
Output: Clear requirements vs. capability gap analysis
5. Cost Comparison Model
Analyze total cost of ownership for each vendor.
Inputs:
- Monthly API costs (at different volume levels)
- Infrastructure costs (if self-hosted)
- Support costs (premium support, training)
- Integration costs (SDKs, frameworks, tooling)
- Team time (training, maintenance, optimization)
Output:
- 12-month and 3-year cost projection
- Cost per unit of work (e.g., cost per inference, per line of code)
- ROI comparison
Example:
| Vendor | Year 1 Cost | Year 2 Cost | Year 3 Cost | Cumulative |
|---|---|---|---|---|
| OpenAI | $180K | $150K | $150K | $480K |
| Anthropic | $120K | $110K | $110K | $340K |
| $85K | $80K | $80K | $245K | |
| Self-hosted | $200K | $60K | $60K | $320K |
(Self-hosted high Year 1 due to setup; lower ongoing due to no per-token costs)
6. Security & Compliance Audit Template
Detailed security assessment for compliance-critical orgs.
Covers:
- Data protection (encryption, key management)
- Access controls (authentication, authorization)
- Audit logging (what’s logged, retention)
- Incident response (breach notification, remediation)
- Third-party audits (SOC 2, ISO 27001)
- Regulatory compliance (HIPAA, GDPR, SOX, etc.)
- Contractual security terms (liability, indemnification)
Use when: Evaluating for healthcare, finance, or other regulated industry
Output: Security gap analysis and compliance readiness
7. Support & SLA Template
Evaluate vendor support capabilities and reliability.
Covers:
- Support channels (phone, email, chat, dedicated account manager)
- Response times (critical, high, medium, low priority)
- Uptime SLA (guarantee, penalty for breaches)
- Training & onboarding (available, costs)
- Documentation quality (API docs, guides, examples)
- Community support (forums, Stack Overflow, etc.)
- Professional services (consulting, custom training)
Example SLA comparison:
| Metric | OpenAI | Anthropic | |
|---|---|---|---|
| Critical response | 2 hours | 1 hour | 30 min (Enterprise) |
| Uptime SLA | 99% | 99% | 99.95% (Enterprise) |
| Dedicated account mgr | $500K+ ARR | Enterprise only | Yes (Enterprise) |
| Email support | Yes | Yes | Yes |
| Phone support | Enterprise | Enterprise | Yes (Enterprise) |
Using Templates
Step 1: Choose a Template
- Go to Vendor Evaluation → Templates
- Browse available templates
- Click Preview to see full content
- Click Use This Template
Step 2: Customize (Optional)
- Click Customize
- Edit questions/sections for your needs
- Add company-specific requirements
- Save as custom template (reusable for future evaluations)
Step 3: Distribute to Vendors
- Click Share with Vendors
- Enter vendor email addresses
- Vendors receive link, complete questionnaire
- Responses sent to you
Step 4: Compare Responses
- View responses side-by-side
- Score each vendor on key questions
- Generate comparison report
- Present to stakeholders
Customization Examples
Example 1: Healthcare RFP
Start with LLM Provider RFP template. Customize:
- Add HIPAA compliance questions (BAA required? Encryption standard?)
- Add healthcare-specific use cases (patient documentation, diagnosis support)
- Add data residency requirement (must be US-only for HIPAA)
- Add audit requirements (what audits provided? frequency?)
Example 2: Finance Vendor Scorecard
Start with Quick Vendor Scorecard. Customize:
- Add SOX compliance questions
- Add financial data security requirements (encryption standard, audit logging)
- Add cost analysis (per-transaction cost model)
- Add PCI DSS compliance if handling payment data
Example 3: Startup Cost Comparison
Start with Cost Comparison Model. Customize:
- Adjust volume projections to match your growth plan
- Add infrastructure costs (if considering self-hosted)
- Add team time estimates (how long to integrate, maintain)
- Compare ROI if using cheaper model vs. best-quality model
Sharing & Collaboration
Share with Team
- Click Share on a template
- Invite team members (Finance, Security, Product, Engineering)
- Set permissions (View, Edit)
- Team members collaborate on scoring/analysis
Generate Reports
Export evaluation results as:
- PDF — Printable report with scores, recommendations, cost analysis
- CSV — Raw data for further analysis
- Slide deck — Share findings with executives
Template Best Practices
1. Align on Evaluation Criteria First
Before sending RFP to vendors:
- Agree on weighted criteria (security 30%, cost 20%, features 25%, etc.)
- Define scoring rubric (what earns 90 vs. 70 vs. 50?)
- Get buy-in from all stakeholders (Finance, Security, Product, Tech)
2. Send Same Template to All Vendors
Ensures apples-to-apples comparison. Don’t customize per vendor (shows bias).
3. Ask Vendors to Verify Responses
Vendors sometimes give generic answers. Follow up:
- “You said you offer SOC 2 — can you provide audit report?”
- “You claim 99.9% SLA — what’s the penalty for breaches?”
- “You list HIPAA as available — is it included or extra cost?“
4. Reference Check with Existing Customers
Ask vendor for 3-5 customer references. Call and ask:
- “How reliable is their service?”
- “How good is their support?”
- “Any surprising costs?”
- “Would you recommend them?”
Related Topics
- Vendor Scoring — Standard vendor evaluation scores
- Custom Criteria — Build your own evaluation framework
- Tools - Comparison — Built-in vendor comparison tool
Next Steps
- Identify 3-5 vendor finalists
- Choose appropriate template (RFP for deep eval, scorecard for quick screening)
- Customize for your needs
- Send to vendors for completion
- Score responses and compare
- Reference check with existing customers
- Present recommendation to stakeholders