evaluate

Create vendor evaluation framework and score vendor proposals

Vendor Evaluation

Create evaluation frameworks and score vendor proposals against requirements.

Command

arckit evaluate "<project ID or category>"

Description

Generates vendor evaluation frameworks with scoring criteria, or scores specific vendor proposals. Ensures objective, requirements-driven vendor selection with full audit trails.

Arguments

• Project ID or category: Project identifier or evaluation category (e.g., ‘001’, ‘CRM platforms’)

When to Use

• Before receiving vendor proposals (create framework)
• After receiving proposals (score vendors)
• For comparing multiple vendors
• To ensure objective vendor selection
• For audit trail requirements

Required Context

• Requirements (ARC--REQ-.md) - MANDATORY
• Architecture Principles (ARC-000-PRIN-*.md) - MANDATORY
• SOW (Statement of Work) - Recommended
• DOS (DOS Requirements) - Recommended
• G-Cloud Search (ARC--GCLD-.md) - Recommended

Task Modes

Mode A: Create Evaluation Framework

Generates evaluation criteria before receiving proposals:

Mandatory Qualifications (Pass/Fail):

• Required certifications (PCI-DSS, ISO 27001, SOC 2)
• Minimum experience requirements
• Financial stability
• Reference requirements

Scoring Criteria (100 points):

• Technical Approach (35 points): Solution design, architecture alignment
• Project Approach (20 points): Methodology, risk management, QA
• Team Qualifications (25 points): Experience, composition, certifications
• Company Experience (10 points): References, industry expertise
• Pricing (10 points): Cost competitiveness, value for money

Mode B: Score a Vendor

Scores a specific vendor proposal:

1. Validates mandatory qualifications
2. Scores each category with justification
3. References specific requirement IDs
4. Documents strengths and weaknesses
5. Provides recommendation

Mode C: Compare Multiple Vendors

Compares all evaluated vendors:

1. Side-by-side scoring comparison
2. Strengths/weaknesses matrix
3. Risk comparison
4. Cost comparison
5. Final recommendation with rationale

Output

Evaluation Framework: projects/{project}/ARC-{PROJECT_ID}-EVAL-v1.0.md

Vendor Scoring: projects/{project}/vendors/{vendor}/evaluation.md

Vendor Comparison: projects/{project}/ARC-{PROJECT_ID}-VEND-v1.0.md

Example Usage

Create Framework

arckit evaluate "Payment gateway project"

Generates:

• Mandatory quals: PCI-DSS certified, 5+ years experience
• Scoring criteria weighted by requirement priorities
• Evaluation process and timeline

Score Vendor

arckit evaluate "Score Acme Payment Solutions for project 001"

Generates:

• Technical: 28/35 (strong PCI-DSS, limited cloud)
• Team: 20/25 (good references)
• Total: 76/100
• Recommendation: APPROVED WITH CONDITIONS

Compare Vendors

arckit evaluate "Compare all vendors for payment gateway"

Generates:

• Acme (76), BestPay (82), CloudPayments (71)
• Recommendation: BestPay - highest score, best cloud experience

Scoring Requirements

Objective Scoring

• All scores require specific justification
• No arbitrary numbers
• Reference requirement IDs (e.g., “Meets NFR-S-001”)
• Document any assumptions

Mandatory Qualifications

• Pass/fail criteria
• Missing any = disqualified
• Security and compliance violations typically blocking

Transparency

• Document conflicts of interest
• Record dissenting views
• Maintain audit trail
• Keep vendor proposals confidential

Related Commands

• arckit sow - Generate SOW with evaluation criteria
• arckit dos - Create DOS with evaluation approach
• arckit gcloud-clarify - Analyze G-Cloud service gaps
• arckit requirements - Source requirements for evaluation

Next Steps

After evaluation:

1. Review scores with stakeholders
2. Validate scoring consistency
3. Conduct vendor demos/interviews
4. Check references
5. Make final selection
6. Document decision rationale
7. Award contract