Vendor Tool Evaluation Without a Given Rubric

TechBridge needs to choose between 3 observability vendors. No rubric is provided — the manager must build their own evaluation framework from scratch, rather than accepting an AI-generated one uncritically.

# Aplly.xyz Case Study Submission

## Title
Vendor Tool Evaluation Without a Given Rubric

## Type
Business Strategy

## Difficulty
Intermediate

## Estimated Time
60 minutes

## Overview
TechBridge needs to choose between 3 observability vendors. No rubric is provided — the manager must build their own evaluation framework from scratch, rather than accepting an AI-generated one uncritically.

## Case Details

Function Focus: Framework construction, weighted decision-making, stakeholder reconciliation

Scenario:
Vendors A, B, and C differ on price, feature depth, support quality, and integration effort. Three stakeholders have conflicting priorities: Engineering wants feature depth, Finance wants lowest cost, and Ops wants reliability/support responsiveness. No one has handed the manager a scoring rubric — they must build one and justify it.

Dataset Structure:
- Vendor feature comparison table (price, feature score, support SLA, integration effort)
- Three stakeholder priority statements (quoted, conflicting)

Tasks:
1. Build your own weighted scoring framework from scratch — define the criteria and assign weights, and justify each weight in writing
2. Score all 3 vendors manually against your framework
3. Explicitly reconcile the conflicting stakeholder priorities in your weighting rationale — don't just average them silently
4. Only after finishing your own framework: check it against an AI-suggested rubric for the same decision, and note where they agree or diverge and why

Expected Output:
Custom rubric with justified weights + scored vendor comparison + stakeholder reconciliation note + a short delta analysis against the AI-suggested rubric.

Evaluation Criteria:
Originality and internal soundness of the self-built framework, transparency of weighting logic, and quality of the stakeholder reconciliation reasoning.

## Data Sources

Vendor comparison:

| Vendor | Price/mo ($) | Feature Score (raw, /10) | Support SLA | Integration Effort (days) |
|---|---|---|---|---|
| A | 4,500 | 8 | 4hr response | 5 |
| B | 3,200 | 6 | 24hr response | 2 |
| C | 6,000 | 9 | 1hr response | 8 |

Stakeholder priority statements:
> Engineering: "We need deep tracing and alerting features — we're growing fast and will outgrow a shallow tool within 6 months."

> Finance: "Budget is tight this quarter. Anything over $4,000/month needs strong ROI justification."

> Ops: "Our last incident took 6 hours to diagnose partly because support was slow. Response time matters more than anything else on this list."

## Solution Frameworks
Weighted decision matrix (self-constructed), multi-stakeholder tradeoff reconciliation

## Solver Guidance & Tutorials
Link to: "Building Decision Frameworks From Scratch" tutorial

## What You'll Learn
- Constructing judgment frameworks instead of consuming pre-made ones
- Reconciling genuinely conflicting stakeholder priorities without hand-waving
- Critically comparing your own reasoning against an AI-generated alternative

## Tags
vendor evaluation, decision frameworks, business strategy, stakeholder management

## Registration Links
- Register as Solver
- Register as Evaluator

Vendor comparison:

| Vendor | Price/mo ($) | Feature Score (raw, /10) | Support SLA | Integration Effort (days) |
|---|---|---|---|---|
| A | 4,500 | 8 | 4hr response | 5 |
| B | 3,200 | 6 | 24hr response | 2 |
| C | 6,000 | 9 | 1hr response | 8 |

Stakeholder priority statements:
> Engineering: "We need deep tracing and alerting features — we're growing fast and will outgrow a shallow tool within 6 months."

> Finance: "Budget is tight this quarter. Anything over $4,000/month needs strong ROI justification."

> Ops: "Our last incident took 6 hours to diagnose partly because support was slow. Response time matters more than anything else on this list."

Weighted decision matrix (self-constructed), multi-stakeholder tradeoff reconciliation