Will your AI system survive its first year?
Twelve questions about the part below the waterline: secrets, access, logging, error paths, rollback, review gates, testing, and vendor change. The difference between a system and a thing that looked finished at the demo.
Answer for one system you are running or about to ship. Not sure counts as a gap, and it is the most honest answer on here.
This is a self assessment, not an audit and not a certification. It sees what you tell it about one system on one day. A clean run means the obvious things are in place, which is worth knowing and is not the same as being secure. Anything load bearing deserves a real review by somebody who reads the code.
SECRETS
Are the API keys and passwords out of the code, out of the chat history, and in a secrets manager?
SECRETS
Could you rotate every credential the system uses in one afternoon, without breaking it?
ACCESS
Does every person and every connected service get only the access it needs, under its own identity?
ACCESS
When somebody leaves, does their access die the same day, on a checklist somebody actually runs?
LOGGING
Is there a log of every action the system took, readable by somebody who did not build it?
ERROR PATHS
When a step fails, does the system do something you chose on purpose: stop, retry, or alert?
ROLLBACK
Can a bad run be undone without a developer and a lost weekend?
HUMAN REVIEW
Do the dangerous actions, meaning payments, mass sends and deletes, wait for a human before they fire?
EVALUATION AND TESTING
Was it tested against the ugly inputs: the blank field, the duplicate, the weird date, the double submit?
EVALUATION AND TESTING
When something it depends on updates, does anybody re-test it, or do you find out from a customer?
VENDOR AND MODEL CHANGE
If the model or the vendor changed behavior tomorrow, would anything in your system notice and tell you?
VENDOR AND MODEL CHANGE
Is there a written plan for switching models or vendors that does not amount to a rebuild?
The twelve questions
- Are the API keys and passwords out of the code, out of the chat history, and in a secrets manager?
- Could you rotate every credential the system uses in one afternoon, without breaking it?
- Does every person and every connected service get only the access it needs, under its own identity?
- When somebody leaves, does their access die the same day, on a checklist somebody actually runs?
- Is there a log of every action the system took, readable by somebody who did not build it?
- When a step fails, does the system do something you chose on purpose: stop, retry, or alert?
- Can a bad run be undone without a developer and a lost weekend?
- Do the dangerous actions, meaning payments, mass sends and deletes, wait for a human before they fire?
- Was it tested against the ugly inputs: the blank field, the duplicate, the weird date, the double submit?
- When something it depends on updates, does anybody re-test it, or do you find out from a customer?
- If the model or the vendor changed behavior tomorrow, would anything in your system notice and tell you?
- Is there a written plan for switching models or vendors that does not amount to a rebuild?
Find out what is actually broken before you spend another dollar on AI.
Two weeks, fixed fee, fixed start date. A map of what you run, a ranked list of where AI actually holds with a dollar figure on each, and a fixed price plan to build it. You own the document either way.
Read it. If you do not agree we found at least three things worth fixing, tell us and we refund the fee. You keep the document.
$5,000. TWO WEEKS. THE FEE COMES OFF THE BUILD IF YOU START WITHIN SIXTY DAYS.
WHO THIS IS FOR
A company of roughly ten to two hundred people, already running a CRM or operational system, with revenue coming through it and at least one process held together by hand. That includes agencies, whether the system is for your own operation or for a client's. Pre revenue, or want a chatbot on a marketing site? Wrong firm, and we will say so on the first reply.
- $5,000
- FIXED FEE. IT COMES OFF THE BUILD IF YOU START WITHIN SIXTY DAYS.
- 2
- WEEKS, ON A FIXED START DATE.