AI Collab Literacy (ACL) — Free Sample

1. You receive a fuzzy product ask: "The assistant should refuse to give specific medical advice but should be helpful for general health questions." Which deliverable is highest-leverage to author first?

A 100-row graded eval set with edge cases (clear refuse, clear help, ambiguous middle, adversarial), pass-rate threshold defined. A revised prompt with explicit refuse-vs-help instructions and three example pairs. A spec doc describing the desired behavior in prose, escalated to legal review. A list of every medical-advice scenario the team has worried about, no pass/fail criteria.

2. The model produces an output that's superficially correct but wrong on a subtle dimension your customer flagged. What's the highest-value next step?

Author a small adversarial eval probing the specific failure mode + similar-shape inputs to test whether the failure generalizes. File a bug, swap the model version, retest the original input. Add a rule to the prompt explicitly forbidding the subtle failure pattern. Tell the customer the model is working as designed and adjust expectations.

3. A teammate proposes shipping a prompt change because it "feels better" on three example inputs. The team has a 200-item production eval. What's the right response?

Run the eval AND ask the teammate to add their three new examples to the eval; rebuild the rubric if the new examples expose a coverage gap. Run the eval on the new prompt; ship if it improves on the eval, regardless of vibes; revert if it regresses. Trust the teammate's judgment on the three examples and ship. Reject the change because the eval is the source of truth.

4. Engineering says the model can't reliably produce behavior X at acceptable cost. Product says we committed to shipping X next quarter. What's the highest-leverage move?

Author an eval that operationalizes "acceptable" as a pass-rate threshold; if the model's pass rate is below it, surface the gap with evidence and renegotiate scope. Push engineering harder; AI capability is improving fast and the model probably can do it with better prompting. Ship behavior X with caveats and quality monitoring, accepting the risk. Cancel the commitment unilaterally without an eval — engineering's intuition is enough.

5. You're authoring a spec for an LLM-driven feature. Which property is most important for the spec to have?

Verifiability: the spec is paired with an eval the team can run to know whether the model meets it. Completeness: the spec covers every edge case the team has thought of. Brevity: the spec fits on one page. Aspirational framing: the spec describes the ideal user experience without constraining how the model achieves it.

Your directional indicator

This 5-question sample gives a rough indication only. For a valid, verified Skills Passport credential, take the full 40-question assessment — it takes about 30 minutes.

Coming soon

Full assessment launching soon