Designing an Admin Console You Can Safely Give to Support

The first version of every admin console is a page that runs SQL. Someone on the team needs to see why a tenant's invoice is wrong, so they get a query box, and it works. Then support grows, and the query box is still there, and now four people who do not read migration diffs have write access to every row in every tenant.

The failure is not usually malice. It is a support agent pasting a UPDATE subscriptions SET status='active' without a WHERE tenant_id, or resetting a password for the wrong Sarah, or exporting a customer list to a spreadsheet to "check something" and leaving it in Slack. Each one is an ordinary Tuesday mistake with an extraordinary blast radius, because the tool was built for the person who wrote it, not the person who uses it fifty times a day.

Across 77 platforms and 700+ tenant databases, we have landed on a consistent shape for internal tooling: the console exposes named actions, never raw data access; every action is attributable to a human and a reason; and the default posture is read-only with most fields redacted. That sounds restrictive until you watch a support agent resolve a ticket in forty seconds because the one thing they needed was a button instead of a Postgres prompt.

Model the console as an action catalog, not a database browser

The single decision that determines whether your admin console is safe is whether it is built around tables or around tasks. A table-shaped console — tenants, users, subscriptions, each with an edit form — inherits your entire schema as its permission surface. Every column you add is a new thing support can break, and you will not notice until it breaks.

An action-shaped console starts from the ticket queue. Go read the last hundred support tickets and write down what the resolution actually was. In our experience that list collapses to fifteen to thirty distinct operations: resend a verification email, extend a trial by N days, unlock a locked account, void an invoice, force a re-sync of a marketplace listing, re-run a failed import, flip a feature flag for one tenant. Each one becomes a first-class server-side action with typed inputs, validation, and its own permission.

Define them declaratively so the permission model and the audit record come from the same place as the UI:

export const extendTrial = defineAdminAction({
  id: "billing.extend_trial",
  label: "Extend trial",
  roles: ["support_l1", "support_l2"],
  requiresReason: true,
  input: z.object({
    tenantId: z.string().uuid(),
    days: z.number().int().min(1).max(14),
  }),
  // limits are enforced server-side, not in the form
  rateLimit: { perActor: "5/day", perTenant: "2/month" },
  async run(ctx, { tenantId, days }) {
    const t = await ctx.tenant(tenantId);         // scoped connection
    if (t.plan === "enterprise") throw new NeedsApproval("sales_owned");
    return billing.extendTrial(t, days, ctx.actor, ctx.reason);
  },
});

The days.max(14) is the important line. A raw date field lets someone type 2099. A bounded integer means the worst-case mistake costs you two weeks of revenue on one tenant, and the escalation path for anything longer is a human who owns that decision. Bound every numeric input in an admin action. There is always a sane ceiling, and if there is not, the action belongs to engineering.

Three tiers of access, and nobody starts at the top

Role sprawl kills admin consoles. We keep three tiers and resist adding a fourth until a real ticket pattern demands it.

• L1 support — read redacted tenant state, run reversible actions (resend email, unlock account, clear a cache, re-run a sync). Cannot see PII beyond what is needed to confirm identity. Cannot touch money.
• L2 / billing — everything L1 has, plus financial actions with limits: credits under a fixed dollar cap, invoice voids within the current period, plan changes that route through the same proration logic as self-serve. Nothing here bypasses the billing engine.
• Engineering break-glass — direct data access, one tenant at a time, time-boxed, always logged, never the default session. This is the only tier that can run arbitrary reads, and it should be uncomfortable enough to use that nobody uses it casually.

The rule we apply when someone asks for a new admin capability: if the action is reversible and its worst outcome affects one tenant, it can be L1. If it moves money or touches another tenant's data, it is L2 with a cap. If it cannot be expressed as a bounded action at all, it is break-glass and it needs a ticket.

Impersonation is the feature, and it needs guardrails

Support cannot diagnose "the button does nothing" from a database row. They need to see the app as the user sees it, which means impersonation, which means the most dangerous feature in your product. Build it deliberately:

1. Issue a distinct session type. Not the user's token — an impersonation token carrying actor_id, subject_id, reason and a short TTL (we use 30 minutes). Every request logs the actor, not just the subject.
2. Make it visibly different. A persistent banner across the top of the app: "Viewing as sarah@tenant.com — read-only — exit". Support agents forget which tab is which. The banner is the cheapest incident prevention you will ever ship.
3. Default to read-only. Write requests from an impersonated session are rejected at the API boundary unless the action is on an explicit allowlist. Most tickets are diagnosed by looking.
4. Never impersonate into secrets. API keys, saved payment methods, stored documents and export endpoints stay masked under impersonation regardless of role. If the customer needs their own key rotated, that is an action, not a peek.
5. Notify or record. For B2B products, write the impersonation into the tenant's own audit log so their admins can see it. That single feature has closed more security questionnaires for us than any policy document.

The awkward part: read-only impersonation cannot reproduce write bugs. When a checkout fails, someone has to attempt a checkout. Our answer is a scoped write mode that requires an L2 to co-sign and expires in ten minutes, and it is rare enough that the log is short and reviewable. If your product is consumer-scale with a high ticket volume, that co-sign will become a bottleneck and you will need per-action allowlists instead — the EzyEvite side of our portfolio behaves very differently here from the B2B platforms.

Redact by default and make un-redacting an event

Support does not need a full credit card, a full email, or a customer's home address to resolve most tickets. They need enough to confirm they are looking at the right record. So render s••••@tenant.com, •••• 4242, last four of the phone number, and put a "reveal" control next to each field that needs it.

Reveals are logged as actions with a reason. This is not theater. Once reveals are logged, the volume tells you something useful: if agents reveal full email on ninety percent of tickets, your identity-confirmation flow is wrong and you should fix that instead of making them click. If reveals cluster on one agent, you have a conversation to have. You cannot have either conversation if the data was on the page all along.

Do the redaction server-side. A masked field that arrives in the JSON payload and gets hidden by CSS is not redacted, it is decorated, and the browser devtools are one keystroke away.

The audit log is the product, not the paperwork

An admin audit log that records "user updated tenant" is worthless during an incident. What you need six months later, when a customer claims their plan was downgraded without permission, is: who, which tenant, which action id, the exact input payload, the before-and-after of what changed, the reason string, the source IP, and whether it happened inside an impersonated session.

Two structural rules. First, write the audit entry in the same transaction as the change, so you cannot have a mutation without a record. Second, write it somewhere the console cannot delete from — append-only table with no delete grant for the app role, shipped to cold storage on a schedule. An audit log your admins can edit answers no questions.

If the only way to know what support did to a tenant is to ask support, you do not have an admin console, you have a shared password.

Reason strings deserve a note. Free-text "reason" fields degrade into "fixing" within a week. Give a short enum tied to your ticketing categories plus an optional note, and require a ticket reference for anything financial. Now the log joins to the support system and you can answer "show me every credit issued without a ticket" in one query.

Scoping data access when tenants live in separate databases

The console's hardest technical problem is cross-tenant queries. Support asks "which tenants are failing marketplace sync" and the honest answer requires touching every database. On a fleet of per-tenant Postgres instances — the model we operate for several platforms, and the one we discuss in shared schema versus database per tenant — a naive console fans out hundreds of connections and takes your fleet down to answer a support question.

What works: push operational state up. Health, sync status, usage counters, entitlement state and billing summaries get written to a central control-plane database as they change. The console reads the control plane for anything list-shaped or aggregate, and only opens a tenant connection when an agent has drilled into one specific tenant. That connection is scoped, short-lived, and tagged with the actor so slow-query logs point at a human.

This also fixes the accident case. If the console physically cannot hold connections to more than one tenant at a time, a mistyped query cannot span tenants. Architecture beats a WHERE clause you have to remember.

Where this advice does not apply

If you are pre-revenue with three engineers and no support team, building this is the wrong use of a month. The query box is fine when the only people with access are the people who wrote the schema and the blast radius is your own ten pilot accounts. Ship the product.

The trigger to build it properly is the first non-engineer who needs access. That is the moment the cost curve flips, because now you are choosing between a week of internal tooling and an unbounded liability. We treat the admin console as part of the initial architecture for any platform that will have a support function — the same conversation as entitlement gating and tenant provisioning, not a follow-up project.

One more sequencing note from operating Retailer OS: build the actions support asks for after you have watched them work, not before. The list you imagine and the list they need overlap by about half. Sit with the ticket queue for a day, count the resolutions, and build the top ten. The rest can stay break-glass until a pattern proves otherwise — and if you want a second opinion on where to draw those lines for your platform, that is the kind of thing we do.