AI Visibility Software Security: The 34-Question Vendor Review for IT and Procurement

An AI visibility software security review verifies how a brand-monitoring platform collects, transmits, stores and deletes your prompts, connected-system tokens and captured AI answers — including every model provider, proxy network and scraping intermediary it touches on your behalf. Standard SaaS questionnaires miss most of that, because they assume the vendor is the only party processing your data. In an AI monitoring workflow, it never is.

This checklist exists because marketing teams keep buying these tools on a credit card, and security teams keep discovering them six months later during an access review. What follows is the question set that surfaces real risk: 34 questions across seven control areas, the data-flow map to run them against, the weak answers that should stop a deal, a weighted scoring model that ends in a decision instead of a stalemate, and the procurement sequence that keeps the whole thing inside three weeks.

What does an AI visibility software security review actually cover?

It covers four things a normal SaaS review does not:

• The prompt library you send into third-party models every day
• The AI-generated answers the vendor stores on your behalf as evidence
• The chain of model providers, proxy networks and scraping intermediaries acting as undisclosed subprocessors
• The write-level access the tool requests to your search, analytics and CMS systems

Everything else — encryption at rest, SOC 2 Type II, penetration testing cadence, SSO — is table stakes your existing forms already cover. The gap is that most published buyer guides stop at "require SOC 2 Type II and SAML." That certifies the vendor's own control environment. It says nothing about whether your competitive prompt set landed in a consumer ChatGPT account routed through a residential proxy network.

Why AI visibility tools break a standard vendor questionnaire

Three structural differences make the usual form inadequate. Each one changes which questions matter.

The vendor is not the only processor. An ai search monitoring platform runs your prompts against ChatGPT, Gemini, Perplexity, Claude, Copilot, Grok and Google AI Mode — daily, at volume. Every one of those providers processes your text. Your DPA with the vendor is meaningless if the vendor's own upstream terms allow retention or training you never approved.

The sensitive asset is the query, not the customer record. Most privacy reviews hunt for PII fields. Here the crown jewel is the prompt library itself. A well-built prompt set names your competitors, your unreleased category framing, your target buyer segments and the objections you are trying to neutralise. Leaked, it is a readable strategy document — and unlike a password, you cannot rotate it.

The output is unpredictable and stored by default. The platform captures raw model answers as evidence. Those answers can contain third-party personal data, defamatory claims about your brand, or competitor material the model fabricated. You are now storing content you did not author, cannot fully predict, and may have to produce in a dispute.

The seven data flows to map before you ask anything

Map first, question second. Every question below is only useful once you know which flow it applies to. This is the architecture shared by essentially every ai visibility tool on the market, regardless of branding.

#	Data flow	What moves	Sensitivity	Who else can see it
1	Prompt library	Your tracked questions, competitor names, brand terms	High — strategic	Vendor, every model provider queried
2	Engine query layer	Prompts sent to ChatGPT/Gemini/Perplexity and others	High	Model providers, scraping subprocessors, proxy networks
3	Raw answer capture	Full AI responses, cited URLs, competitor mentions	Medium — unpredictable content	Vendor, storage provider
4	Evidence artifacts	Screenshots, HTML snapshots, share links	Medium — often public URLs	Anyone with the link, if unsigned
5	Connected systems	Search Console, analytics, CMS, CRM tokens	High — write access possible	Vendor, any OAuth-scoped integration
6	User and org data	Seats, roles, SSO identities, client accounts	Medium	Vendor, identity provider
7	Reporting exhaust	Exports, scheduled emails, Slack/Teams webhooks	Medium	Recipients, messaging platform

Flows 2 and 4 are where reviews usually fail. Flow 2 because buyers assume "the vendor calls an API" without asking which one. Flow 4 because nobody thinks of a screenshot as a data asset until a client-facing share link turns up in a search index.

Flow 5 is the one that escalates fastest in incident review. A rank tracker holds a read-only Search Console scope. An AI visibility platform with a "publish the fix" feature holds a CMS token that can rewrite live pages — a supply-chain path into your own site, not just a reporting risk.

How does the vendor actually query ChatGPT, Gemini and Perplexity?

This is the single highest-value question in the review, and almost no published checklist asks it. Three collection methods are in production use across the category, with completely different risk profiles.

Collection method	How it works	Who sees your prompts	Primary risk
Official provider APIs	Vendor holds enterprise API contracts and queries models directly	Vendor + named model providers under commercial terms	Retention defaults; whether zero-retention was actually enabled
Headless browser on logged-in accounts	Automation drives consumer or team accounts through a browser	Vendor + provider under consumer terms + proxy vendor	Consumer terms may permit training; terms-of-service exposure; personalised results skew data
Resold scraping APIs	Vendor buys AI-answer data from a third-party scraping provider	Vendor + scraping firm + residential proxy network	Undisclosed fourth parties; no direct contractual path for deletion

None of these is automatically disqualifying. Browser-based collection is often the only way to capture what a real logged-in user sees, including AI Overviews and AI Mode surfaces with no public API. What matters is disclosure and control: the vendor must name the method per engine, name every intermediary, and state whether your prompt text is isolated per customer or pooled across their whole book of business.

Pooling is the quiet one. If a vendor deduplicates identical prompts across customers to cut model costs, your query results are shared infrastructure. Ask directly. The honest answer is usually "yes for generic industry prompts, no for brand-specific ones" — which is fine, once it is written down and scoped.

One practical tell: ask for a single-prompt trace. Pick one tracked prompt and ask the vendor to walk the exact path — which account executed it, in which region, through which intermediary, where the response landed, and which log lines exist to prove it. Vendors running official APIs answer this in one screen share. Vendors reselling scraped data usually need to "check with engineering," because the answer involves a company they do not want named.

The 34-question AI visibility software security checklist

Run these in order. Send sections 1–4 by email before you book a demo; the response quality tells you whether the vendor has a security function or a support inbox with a template.

Section 1 — Prompt data and query library (6 questions)

1. Where is prompt text stored, in which region, and is it encrypted at rest with customer-scoped keys?
2. Which collection method is used per engine — official API, browser automation, or resold scraping?
3. Are prompts deduplicated or pooled across customers, and can pooling be disabled for our account?
4. Do any staff or support tools have plaintext read access to our prompt library? Under what approval?
5. Is prompt text ever included in your own product analytics, error monitoring or LLM-based feature development?
6. If we delete a prompt, is it removed from historical run records, or only from the active tracking set?

Good answer to #6: the vendor distinguishes deleting a tracked prompt from purging its historical results, and offers both. A vendor that treats them as one operation has not thought about audit trails.

Question 5 catches a genuinely common leak. Prompt text routinely lands in error-monitoring payloads — a failed engine call serialises the request body into a stack trace, and that trace goes to a third-party observability vendor nobody listed as a subprocessor. Ask specifically whether request bodies are scrubbed before they leave the application.

Section 2 — Customer and business data (4 questions)

7. Does the platform ingest any customer PII — CRM exports, support tickets, review text, named accounts?
8. What is the minimum data set required for the product to function, and what is merely optional enrichment?
9. Are competitor names, internal codenames or unreleased product names treated as confidential fields?
10. Do connected integrations request read-only scopes, and can you provide the exact OAuth scope list per integration?

Question 10 catches scope creep. Several platforms request write access to publishing systems so they can "fix" content automatically. That is a different risk class than reading rankings, and it belongs in front of your change-management process, not a marketing budget line. Insist on the literal scope strings — https://www.googleapis.com/auth/webmasters.readonly and .../auth/webmasters differ by one word and by the ability to submit sitemaps and alter site settings.

Section 3 — Retention, deletion and residency (5 questions)

11. What is the default retention period for prompts, raw answers, screenshots and logs — stated separately?
12. Can retention be configured per data type, and what is the shortest supported window?
13. What is the contractual deletion SLA after termination, and does it cover backups?
14. In which regions is data stored and processed, and is EU or UK residency available without a custom contract?
15. Which upstream providers retain copies you cannot delete, and for how long?

Question 15 is where most vendors go quiet. It has a knowable answer. Under OpenAI's documented API data controls, API inputs and outputs are retained for up to 30 days for abuse monitoring unless the account is approved for Zero Data Retention, and API data is not used to train models unless the customer opts in. A competent vendor tells you which of their upstream accounts have ZDR enabled and which do not. A vendor who says "we delete everything immediately" without qualifying the upstream layer is describing their own database and calling it the whole system.

Retention has an asymmetric failure mode worth naming: your deletion request typically reaches layer one only. Four layers, four separate clocks:

1. Vendor application database — the easy one, usually honoured within days
2. Vendor backups — commonly 30–90 days of rolling snapshots that no delete request touches
3. Model-provider abuse logs — up to 30 days on standard OpenAI API terms, zero under ZDR, unknown under consumer terms
4. Scraping subprocessor caches and support-ticket attachments — the layer with no contract between you and the holder

Write the deletion SLA against all four, or you have contracted for one quarter of a deletion.

Section 4 — Subprocessors and model-provider exposure (5 questions)

16. Provide the complete subprocessor list, including model providers, proxy networks and scraping vendors.
17. What notice period do we get before a new subprocessor is added, and can we object?
18. Which commercial terms govern each model-provider relationship — enterprise, team or consumer?
19. Are any subprocessors located outside our required data-residency region?
20. Do you hold a DPA with each subprocessor, and can we see the flow-down clauses?

Questions 16 and 17 are not preferences. Under GDPR Article 28, a processor cannot engage another processor without prior specific or general written authorisation from the controller, and where general authorisation applies the processor must notify the controller of intended changes and give it the opportunity to object. A vendor whose subprocessor list omits the model providers it queries daily has either not read Article 28 or is hoping you have not.

Practical test: compare the vendor's public subprocessor page against the engine list on their own pricing page. If the marketing site advertises coverage of eight engines and the subprocessor page names two cloud hosts, the list is incomplete by the vendor's own admission. This mismatch is common enough that it belongs in your first email, not your third.

Section 5 — Access control, identity and provisioning (5 questions)

21. Is SAML SSO available on our plan tier, or gated behind an enterprise upgrade?
22. Is SCIM provisioning and — more importantly — automatic deprovisioning supported?
23. What roles exist, and can a role be restricted to a single brand, market or client workspace?
24. Can report share links be forced to require authentication, org-wide?
25. What is your internal access model for customer data, and is production access broken-glass and logged?

Question 24 matters disproportionately for agencies. Public share links are a beloved client-reporting feature and a standing exposure for anyone tracking multiple competing brands in one account — the failure is not a breach, it is a link forwarded to the wrong client that reveals which competitors you monitor for whom. Agencies weighing operating models will find this interacts with everything else in the agency versus in-house decision, because workspace isolation is what makes a shared toolset defensible.

Question 22 is the one that quietly fails audits. Seats in marketing tools outlive employment more often than seats in IT-managed systems, because nobody offboards a tool procurement never registered. Ask what happens to a user's active session and API keys at deprovision time, not just to their login.

Section 6 — Audit logs and evidence artifacts (4 questions)

26. Which events are logged — logins, prompt edits, exports, share-link creation, integration changes?
27. Can audit logs be exported via API or streamed to our SIEM, and what is the log retention period?
28. Are stored screenshots and HTML snapshots served from signed, expiring URLs or from public object storage?
29. Is the scoring methodology documented well enough that an internal auditor can reproduce a reported number?

Question 29 is a security question disguised as a methodology question. If a platform reports an ai share of voice figure nobody outside the vendor can reproduce, you cannot audit it, defend it in a board pack, or detect when it silently changes. A weighting tweak shipped in a release note becomes a 12-point "improvement" nobody earned. That is why the underlying AI visibility score formula should be published rather than hidden — an unauditable metric is an unauditable system.

Test question 28 in ninety seconds: open a stored screenshot in a private window with no session. If it renders, it is public. If the URL has no expiry parameter, it is public forever.

Section 7 — Corporate security posture and incident response (5 questions)

30. Provide your SOC 2 Type II report and, if the audit window has lapsed, a bridge letter.
31. When was your last third-party penetration test, and can we see the summary and remediation status?
32. What is your contractual breach notification window, in hours?
33. Do you carry cyber liability insurance, and at what limit?
34. Is there a documented business continuity plan, and what happens to our historical data if you are acquired or shut down?

Question 32 has a regulatory floor worth quoting back. GDPR Article 33 requires a controller to notify the supervisory authority within 72 hours of becoming aware of a personal data breach, and Article 33(2) requires the processor to notify the controller "without undue delay." "Without undue delay" is not a number. If your vendor's notification clock consumes 48 of your 72 hours, your own compliance obligation is already impaired. Put 24 hours in the MSA.

Question 34 deserves more weight than it usually gets in a category this young. Historical AI visibility data is not re-collectible — you cannot go back and ask ChatGPT what it said about you last March. Contract for an export path, in a documented format, before you need one.

Red-flag answers and what they actually mean

Vendors rarely refuse to answer. They answer adjacently. This table maps the most common evasions to their translation.

You asked	Weak answer	What it usually means	What to demand instead
How do you query each engine?	"We use industry-standard methods."	Browser automation or resold scraping they would rather not name	A per-engine table: engine, method, account type, intermediary
Full subprocessor list	"AWS and Google Cloud."	Model providers omitted entirely	Named model providers with commercial tier per provider
Retention period	"We retain data for as long as your account is active."	No configurable retention exists	Per-data-type windows plus a shortest supported setting
Are prompts used to improve your product?	"We may use aggregated, anonymised data."	Prompt text feeds internal model or feature work	Written scope of what "aggregated" excludes
SSO availability	"Available on Enterprise."	SSO is a paywalled upsell, not a control	SSO at the tier you are actually buying
Screenshot storage	"Stored securely in the cloud."	Public bucket with guessable paths	Signed URLs with a stated expiry
Breach notification	"As required by law."	No contractual commitment	A number in hours, in the MSA
Deletion on termination	"We delete your data."	Application database only	Deletion SLA covering backups and named upstream layers
Data residency	"Hosted in the EU."	Storage is EU; the query path is not	Residency stated for storage and collection
Certification status	"SOC 2 in progress."	No audited control period yet exists	Report with dates, or a compensating-controls memo

How to score the review without a pass-or-fail deadlock

Binary reviews fail in this category. Almost no vendor scores clean, and a pure pass/fail gate either blocks every tool or waves through the one with the best-looking PDF. Weight by blast radius instead.

Score each section 0–3 (0 = no answer or evasion, 1 = partial, 2 = adequate, 3 = documented and verifiable), then apply these weights:

Control area	Weight	Rationale
Prompt data and query library	20	Strategic exposure, irreversible if leaked
Subprocessors and model exposure	20	Parties you never contracted with
Customer and business data	15	Direct regulatory exposure
Retention, deletion, residency	15	Drives DSR and audit obligations
Access control and identity	15	Determines internal blast radius
Audit logs and evidence artifacts	10	Determines whether you can detect anything
Corporate posture and incident response	5	Important, but well covered by existing forms

Convert to a percentage of 300 possible weighted points. Above 80% is approvable with standard contract terms. 65–80% is approvable with specific redlines and a 90-day remediation commitment. Below 65% should not proceed regardless of how good the demo was.

Apply two hard gates on top of the score. A zero in prompt data or in subprocessors is a stop, no matter what the total says — those two sections are where the unrecoverable failures live. Everything else can be fixed with a contract clause or a configuration change after signature; a leaked prompt library and an unnamed fourth party cannot be walked back.

One calibration note: a young vendor scoring 70% with precise, self-critical answers is a better risk than an incumbent scoring 82% on boilerplate. Score the specificity, not the confidence.

Where the bar moves by industry and geography

Regulated buyers need more than this baseline. Fintech, healthcare and legal teams are accountable not only for how the tool handles data but for what the AI answers say about them, which turns monitoring into a records-retention obligation rather than a marketing report. That calls for a documented, timestamped capture chain and a defined escalation path — the substance of a compliance-grade AI answer monitoring workflow.

Multi-region programmes add a second dimension. If you track prompts in German, Japanese and Portuguese, the collection layer is hitting engines from multiple geographies, sometimes through region-specific proxies. Data residency then applies to the query path, not only the storage bucket. A vendor can hold data in Frankfurt while executing every query from a US IP under US-tier account terms, and still say "EU hosting" without lying. Ask where the query executes, not where the row lands.

Coverage scope changes the exposure surface too, which is why prompt-set sizing is a security decision as much as a budget one — a 2,000-prompt library across eight engines multiplies both the strategic value of a leak and the number of upstream parties involved. Decide how many prompts to track with that in view rather than buying the largest tier available.

A procurement sequence that avoids a six-week stall

Order matters more than thoroughness. Most reviews stall because the security questionnaire arrives after the demo, when marketing has already picked a winner and every finding reads as an obstruction.

1. Before the demo (day 0). Email sections 1 and 4 — prompt data and subprocessors, eleven questions total. Response time and specificity are your first signal. Vendors with a security function answer in three business days with a document; vendors without one answer in two weeks with adjectives.
2. During the demo (week 1). Have IT in the room. Ask for four things live: an audit-log export, a share-link permission setting, the exact OAuth scopes per integration, and the single-prompt trace described above. Live beats roadmap.
3. Document review (week 2). SOC 2 Type II plus bridge letter, penetration test summary, subprocessor list, DPA with standard contractual clauses where relevant, and the retention configuration options in writing.
4. Redlines (week 3). Four clauses do most of the work: subprocessor change notice with a right to object, deletion SLA covering backups and named upstream layers, breach notification in hours, and a data export right that survives termination or acquisition.

Teams running this as part of a wider programme usually fold it into the same document set as their AI visibility platform RFP requirements, so security scoring and capability scoring land in one comparison sheet rather than two competing ones. Where an entire function is being stood up, security belongs earlier still — in the tooling layer of an enterprise GEO programme's people, process and data design.

Frequently asked questions

Is SOC 2 Type II enough to approve an AI visibility tool?

No. SOC 2 Type II attests that the vendor operated its own stated controls over an audit window. It says nothing about which model providers receive your prompts, what those providers retain, whether browser automation runs through third-party proxies, or whether share links are public. Treat it as a prerequisite, then run the subprocessor and prompt-data sections separately.

Do our prompts get used to train AI models?

It depends on which commercial tier the vendor uses upstream. Enterprise and API relationships generally exclude training by default — OpenAI documents that API data is not used for training unless the customer opts in. Consumer-tier accounts driven by browser automation fall under different terms. Ask the vendor to state the account type per engine in writing, and treat "we don't train on your data" as an answer about the vendor's own models, not the providers it queries.

What retention period should we require for prompt and answer data?

Set it by purpose. Trend reporting on ai citations and brand mentions in ChatGPT needs 12–24 months of aggregate results to stay useful. Raw prompt text and full-answer transcripts rarely need more than 90 days at the record level unless a specific compliance obligation applies. Requiring separate windows per data type is the single most effective retention redline available.

Who should own this review — security, legal, or marketing?

Marketing owns the requirement, security owns the verdict, legal owns the contract clauses. The failure mode is marketing running the whole thing and discovering the gaps at renewal. Bring security into the first demo; a 45-minute early meeting reliably costs less than a retroactive review of a tool already holding two years of llm brand tracking data. Where that ownership line sits depends on how the function is staffed, which is the subject of who should own AI search visibility.

How does this differ from reviewing a normal SEO rank tracker?

A rank tracker reads public search results and stores positions. An ai search monitoring platform transmits your strategic question set into multiple third-party AI systems every day, stores unpredictable generated content, and often requests write-capable integrations. The stored output can contain material about people and companies who never consented to be in your systems. Same category of product, materially different data-protection footprint.

Can we run this review on a free trial before purchasing?

Yes, and you should — but treat the trial as live production data. Trial accounts frequently run on shared infrastructure with pooling enabled and retention defaults nobody configured, so seed the trial with a decoy prompt set rather than your real competitive question library. If the vendor cannot delete the trial data on request and show you the confirmation, that answers question 13 before you ever send it.