What should I ask an AI vendor about their security practices?

Ask about build pipeline hygiene, secret scanning, incident disclosure history, customer notification processes, and what telemetry the tool sends home. Vendors with mature security programs can answer these directly. Evasion is itself a signal.

How do I assess vendor supply chain risk for an AI developer tool?

Review the published package contents, check whether the vendor has a disclosed security program or bug bounty, and look for any prior CVEs. For tools installed as CLI packages, examine what ships in the artifact and whether unnecessary files are included.

Is vendor security assessment realistic for a small team?

A lightweight assessment takes less than 30 minutes. Start with the vendor's security page, search their CVE history, and send a short written questionnaire. You do not need a full procurement process — you need enough signal to make an informed decision.

What does the Claude Code source leak tell us about vendor due diligence?

It shows that even well-resourced vendors make basic operational security mistakes, such as shipping source maps in production builds. It reinforces the need to assess vendor release process maturity, not just vendor reputation.

Where should we record answers and decisions?

Use your central AI tool register and link to vendor files; pair with [AI vendor due diligence](/blog/ai-vendor-due-diligence-30-minutes) for lightweight scoring.

AI Developer Tool Vendor Security — 10 Ques…

When an AI developer tool ships a bad npm artifact, the failure mode is boring — and expensive — at the same time: a skipped checklist item becomes public source, and your team must decide whether the AI developer tool still fits your risk bar. These ten questions surface maturity before adoption, not after incident firefighting.

Key Takeaways

Inspect the shipped artifact yourself; the AI developer tool package is the ground truth.
Written answers beat verbal hand-waves; store them beside your AI tool register.
Pair vendor screening with incident response for AI vendors and governance for hidden capabilities.
Keep authority references handy when executives ask “is this enough?” — see References.
Link evaluations to the vendor evaluation checklist and reuse the scoring cadence from the site's 30-minute vendor due diligence article (paths /blog/ai-vendor-evaluation-checklist and /blog/ai-vendor-due-diligence-30-minutes).

Summary

AI developer tool diligence is a pattern, not a project: quick public research, a short questionnaire, then a decision recorded in your register. The questions below map to concrete artefacts (CI logs, advisories, SBOM claims) so lean teams can move without slowing delivery.

Governance Goals

Make adoption decisions reversible with documented evidence.
Keep security, legal, and engineering aligned on what “good enough” means for an AI developer tool.

Risks to Watch

Artifact bloat (source maps, debug files) expanding attack surface.

Silent telemetry that conflicts with data-class rules.

Opaque dependency trees nobody monitors after install.

Disclosure culture that hides operational mistakes until researchers publish.

Controls for every AI developer tool rollout (What to Actually Do)

Require artifact review before production keys touch the AI developer tool.

Demand CI controls answers with enough specificity to be testable.

Instrument procurement with the published evaluation checklist on this site (see /blog/ai-vendor-evaluation-checklist).

Route weak answers into elevated review using Shadow AI guidance when tools spread unchecked.

Checklist (Copy/Paste)

Tarball or installer inspected locally; unexpected files logged.
Secret scanning question answered with tool names, not platitudes.
Advisory channel subscribed; sample notification reviewed.
SBOM or dependency policy received or explicitly declined with rationale.
Result captured in tool register with owner + date.

Implementation Steps

Desk research (30m). Pull CVE history, npm/GitHub advisories, and prior incidents.
Send questionnaire (10m). Paste the 10 questions; request written replies.
Score responses. Use the due diligence scoring thresholds from /blog/ai-vendor-due-diligence-30-minutes.
Decision + record. Approve, approve with constraints, or defer; note data classes allowed.
Revisit quarterly via usage audit workflow.

Frequently Asked Questions

Q: Do we send all ten to every startup vendor?

A: Prioritise questions 1–4 and 8–10 first; they surface the fastest signal. Follow with the remainder before enterprise-wide rollouts.

Q: What if engineering already installed the AI developer tool?

A: Retro-run the same steps; treat gaps as remediation tasks with dates, not as blame.

Q: How strict should we be on SBOM claims?

A: Ask for an export or attestation; if unavailable, require compensating monitoring (dependency alerts) before sensitive data usage.

Q: Can legal own this alone?

A: No — security or senior engineering should co-sign anything touching CLI installs, keys, or CI integrations.

Q: Where does incident history live?

A: Beside the vendor record, linked to Claude Code governance lessons when relevant.

Questions About Build Pipeline and Release Hygiene

Treat this block as the fastest filter for AI developer tool maturity. If a vendor cannot answer these crisply, slow standardisation until they can — the cost of reversing a CLI that already touched production repos is almost always higher than a two-week delay.

1. What does the published artifact actually contain?

Download the package and inspect it. Look for source maps and debug symbols — the class of mistake seen in the Claude Code npm incident.

2. Do you run automated secret scanning in CI?

You want named tools and enforcement, not “we take security seriously.”

3. What does your pre-publish checklist include?

Mature teams can describe who signs off and what is compared against an allowlist.

Questions About Incident Disclosure

4. What security incidents appeared in the last 18 months and how were they disclosed?

Cross-check public advisories; evasion is signal.

5. How do you disclose issues that do not expose user data?

Mature AI developer tool vendors still communicate operational failures clearly.

Questions About Customer Notification

6. How are customers notified when an event affects software they run?

Email, status page, and GitHub advisories are all valid — silence is not.

7. Do you maintain a security advisory channel?

RSS or mailing lists are cheap and tell you how they think about users.

Questions About Supply Chain and Telemetry

8. What data leaves the workstation during normal use?

Map telemetry to your data policy;pair with acceptable use updates if needed.

9. How do you track third-party dependencies (SBOM or equivalent)?

You are evaluating ongoing monitoring, not a one-time spreadsheet.

Questions About Feature Transparency

10. What capabilities exist that are not yet visible to users?

Connect answers to hidden feature governance; undisclosed autonomy or memory modules matter.

What good answers look like (and what dodging sounds like)

Strong AI developer tool vendors name tools, link runbooks, and give timelines in practice. Weak vendors answer only in hollow superlatives (“we take security seriously”) or defer everything to a future legal review. Your goal is not courtroom testimony — it is enough signal to choose data classes, retention defaults, and escalation paths. When answers contradict public artefacts (for example claiming “no telemetry” while the binary clearly phones home), downgrade the vendor and widen monitoring until the mismatch is resolved. Record those moments; they predict how the vendor will behave during an incident.

How to use these AI developer tool questions

Start self-service, send a short written form, then hold a live pass only for nuanced areas (telemetry, data residency). Document outcomes so the next AI developer tool evaluation is faster.

Buying time without buying risk

If engineering wants to move faster than security can review, default to non-production data classes and narrow scopes while questionnaires are in flight. The AI developer tool can still prove value in sandboxes; the mistake is promoting the same install path to regulated data before answers exist. When a vendor delays responses, record the stall in the register and cap spend — ambiguous vendor behaviour is itself a risk signal you may later compare to peer tools.

When answers improve over time

Vendors sometimes graduate from fuzzy to crisp answers as your account matures. Version the questionnaire snapshots in your register so you can see progress. An AI developer tool that improves disclosure quarter-over-quarter may deserve continuity; one that stalls indefinitely does not. Re-run the artifact inspection after major semver bumps — release velocity can silently reintroduce risky files even when the vendor’s security story sounds unchanged.

References

Companion incident analysis: What the Claude Code Source Leak Reveals About AI Tool Governance.
NIST AI Risk Management Framework: https://www.nist.gov/itl/ai-risk-management-framework
EU AI Act portal: https://artificialintelligenceact.eu/the-act/
Operational rhythm: AI acceptable use policy template.