Audit 2026-10-11 (v0.7.0)
On this page
This is a full end-to-end check before deploying v0.7.0 (User Chat, Dev Chat, Sandbox Factory, HF provider, quotas, devbox executor). We checked everything locally first (opennextjs-cloudflare preview with the remote AI binding and local D1), then on production after the deploy (the "Live" section).
Summary#
| Area | Result |
|---|---|
| Pages (home, templates + 10 templates, builder, design, docs + raw md + llms.txt, chat, chat/dev, factory, 404) | 200 / 404 as expected, CSP on every response, no console errors |
API /api/plan |
415/413/400/429 + Retry-After, HF → offline chain, key never echoed |
| API chat | 400 empty, 413 too_long, 400 invalid JSON, 404 foreign or cross-channel conversation, 404 foreign image, 405 on POST /api/chat, 429 burst |
| API attachments | 413 too_large (11 MB), 415 unsupported_type, 503 uploads_disabled (no R2) |
| API factory | 401 no key, 401 key_rejected for an invalid key, 200 config; self-hosted session with a placeholder-only command |
| User Chat end-to-end (local) | Gemma 4, SEA-LION, Qwen Coder, DeepSeek V3 (HF) streamed; FLUX image 332 KB JPEG |
| Dev Chat end-to-end (local) | Sandbox command python3 -c "print(sum(range(10)))" → 45, 70,369 in / 205 out tokens |
| Quotas | unit-tested: 20 msg, 5 images, IP backstop ×3, agent global 30, Bangkok-midnight buckets, refund |
| a11y (axe wcag2a/aa, serious+critical) | chat / dev / factory / templates / builder clean; older issues listed below |
| Mobile 390px + Thai + reduced motion | no horizontal overflow, lang=th, reduced-motion respected |
| docs:check | passed (51 pages) |
| Tests | 180/180 vitest; tsc, eslint clean; cf:build OK |
| Leaked-secret scan | no real secret values in committed files (scanned in-process against env; only fake test keys and redaction regexes) |
| devbox executor | devboxctl status: supervisor running, 0 restarts, executor up since 13:13 (UTC+7) |
| D1 | local at schema 5; remote at 2, with 0003–0005 pending (applied during the deploy below) |
| Secrets (names) | production had OPENAI_API_KEY only → CHAT_COOKIE_SECRET and HF_TOKEN added at deploy (via stdin) |
| Git | feat/d1-chat-schema and feat/chat-orchestrator were unmerged; both merged into main with --no-ff |
Problems found#
Fixed in this release#
- Cloudflare context was imported dynamically in
runtime.tsandrate-limit.ts. After the first request this failed with "Cannot perform I/O on behalf of a different request", so the Workers rate limiter for/api/plansilently fell back to in-memory counters in production. Fixed with a static import andgetCloudflareContext({ async: true }). - CSP blocked Cloudflare Web Analytics, which is injected on the zone, causing console errors on live. We now allow the
https://static.cloudflareinsights.comscript andhttps://cloudflareinsights.comconnect. robots.txtandsitemap.xmlreturned 404. They are now generated bysrc/app/robots.tsandsitemap.ts(68 URLs;/api/disallowed).- The header overflowed at 1024–1279 px after adding the chat links (189 px). Dev Chat, Factory and Design now show from
xlup, the hamburger menu covers up toxl, the palette trigger label only shows at2xl(with an aria-label), and "Sandbox Factory" is shortened to "Factory" in the nav. - The Dev Chat Sandbox panel showed User Chat quotas. It now shows the agent turns left.
- The home page code sample was a scroll region that couldn't take focus (axe
scrollable-region-focusable). It now hastabIndex=0and an aria-label. cloudflare-env.d.tswas regenerated with runtime types, which broke the type check. It is now generated with--include-runtime=false(thecf:typegenscript).- Plan tests depended on the box's
HF_TOKEN. The tests now stubHF_TOKENandOPENAI_FALLBACK, and a test for the HF-first chain was added. - Orchestrator JSONL logs contained environment ids and remote URLs. They are now redacted (
redact_ids) and existing logs were scrubbed. The logs are gitignored.
Open#
- R2 is not enabled on the account. Images are stored inline in D1 (
attachment_blobs, ≤1.5 MB), uploads answer 503uploads_disabled, and the attach button is disabled. - Turnstile is not implemented on the chat pages. Abuse protection relies on the cookie, IP and global quotas plus the burst limiters.
- There is no hard
max_output_tokensin the Agents API. Dev Chat cost is bounded by the global turn cap (30/day) and the $1/day budget estimate. - The PDF spec contained a live OpenAI key and ids. We never used them; the key must be revoked by its owner.
- Docs prose links aren't underlined (axe
link-in-text-block, /docs). This needs a design decision (visual change). - One contrast hit on
/design: a disabled-state hint (text-subtleon surface, 2.51:1). This needs a design token decision. - Factory files created in a user's own sandbox can't be downloaded through us.
- The devbox reboot hook is best-effort (no systemd/cron on the box;
devboxctl ensureruns from the shell profile). The dashboard self-hosted session may expire upstream. - Planner output reports
source: "openai"withmodel: "hf:…"when the plan came from HF. The schema only knows openai/fallback; this is cosmetic. - One flaky test run:
docs-check.test.tsfailed once whilecf:buildwas regenerating the docs bundle at the same time. It passes when run on its own (3/3). - Workers AI in local preview always uses the account's real allowance (the binding is remote).
Costs (this audit)#
- Workers AI: about 8 text replies plus 2 FLUX images, roughly 1–2k neurons, within the 10k/day free allowance.
- HF: a few DeepSeek V3 calls (HF Pro credits).
- OpenAI: 2 Dev Chat turns (about 70k input tokens each, mostly cached) at roughly $0.02–0.15 total. Factory test sessions with no turns cost nothing.
Live (production, https://devstack.bid)#
The deploy ran on 2026-10-11 at about 13:50 (UTC+7). Worker nvx-stack-builder, account 2d92bd5b25768fa9093d6adc0a8887fc, custom domains devstack.bid and www.devstack.bid (unchanged).
- D1 remote: migrations 0003, 0004 and 0005 applied;
schema_version = 5. - Secrets (names):
OPENAI_API_KEY(kept),HF_TOKEN(new, via stdin),CHAT_COOKIE_SECRET(new, freshly generated, via stdin). - Bindings:
AI(Workers AI),DB,NVX_PLAN_LIMITER,NVX_PLAN_AI_LIMITER,NVX_CHAT_LIMITER,NVX_FACTORY_LIMITER,ASSETS,WORKER_SELF_REFERENCE.
| Check | Result |
|---|---|
| All pages | 200 (404 for unknown paths); CSP includes cloudflareinsights; www → 308 to apex |
robots.txt / sitemap.xml |
200 |
| Console errors | 0 on 20 page loads (desktop EN and mobile TH with reduced motion); the only error is the expected 404 resource on /nope |
| axe | only the open items 5–6 remain (docs links, one /design contrast) |
| User Chat (Thai · SEA-LION) | streamed Thai answer, 181 in / 160 out tokens, messagesLeft 19 |
| User Chat image (FLUX) | 453 KB JPEG, owner-only (404 without the cookie), imagesLeft 4 |
| Dev Chat turn | 2 docs lookups and a sandbox command → 3.12.12 plus the fastapi-ai template, 48,042 in / 203 out tokens, agentLeft 4 |
/factory |
loads; GET /api/factory lists models and scopes |
/api/plan |
provider: huggingface, model: deepseek-ai/DeepSeek-V3 |
Screenshots: screenshots/v0.7/live-*.png.
Rollback#
| Version | What |
|---|---|
450890b7-5432-4795-afbc-1c7ebd241bff |
v0.6.1 (D1 binding); the production version before this deploy |
cfe6c97c-3b8d-40c4-8775-3f3f68b82076 |
v0.7.0, first deploy |
Roll back with wrangler rollback <version-id> --name nvx-stack-builder. The D1 migrations are additive (new columns and tables), so old code keeps working with schema 5.
Source: docs/operations/audit-2026-10-11.md · /docs/operations/audit-2026-10-11.md