ident

43.9 KB

1. # ident.worldapi.org
3. ident: the login and identities for all worldapi apps. **`CONCEPT.md` (the creator's) is
4. the source of truth** — read it first; nothing is built that it does not describe.
5. Part of AntColony (byrodin `/CONTAINERS/projects/antcolony/README.md`, ticket #2; built in
6. 6 pieces, tickets #24–#29).
8. Built so far — **piece 1, ticket #24**: the login to ident itself (email one-time code), the
9. account, its identities, its time zone (page `/`). **Piece 2, ticket #25**: apps (page
10. `/apps`), one identity id per app, the login button flow and the code exchange
11. (see "How apps use ident"). **Piece 3, ticket #26**: the identity selector (`<ident-selector>`,
12. `/selector.js`, same section).
13. **Piece 4, ticket ident#6 (mission 013)**: notifications — kinds, the send API, the inbox
14. (`/inbox`) and the per-app page (`/inbox/<connection id>`); see "How apps send notifications".
15. Not yet: DELIVERY — the daily mail at 17:00 and urgent mail (piece 5, ident#7), push (piece 6,
16. ident#8). Nothing is mailed or pushed yet; ident stores what delivery will need.
18. Written in **Hybriel** on **hl:web** (hybriel master, since mission 036), same stack and conventions as
19. `/media/STORAGE/projects/tickets.worldapi.org`.
21. ## Run (dev, Loreana)
23. ```bash
24. cd /media/STORAGE/projects/ident.worldapi.org
25. setsid nohup ./bin/hybriel project.hl > server.log 2>&1 < /dev/null & echo $! > server.pid
26. # stop: kill $(cat server.pid)
27. ```
29. * Port **8351** on 0.0.0.0 — http://100.77.141.84:8351 (tailnet), http://192.168.178.75:8351 (LAN).
30. * The dev watcher is on: saving a `.hl` file reloads; restart after `.env`/binary changes and after
31. changing `styles.hl` root members or `shared/tokens.hl` (the served sheet is not rebuilt).
32. * **The dev server sends REAL mail** (SMTP via mail.byrod.in, `.env` — do not read or print it,
33. it holds the SMTP password). Never trigger an OTP on :8351 in a test; tests start their
34. own servers with `IDENT_MAIL_SINK`.
35. * **Test app** (`testapp/`, :8354): see "How apps use ident" → "Try it".
36. * Config: **`.env`** in the app folder (read from the cwd; the real environment outranks it):
38. | Variable | Default | |
39. |---|---|---|
40. | `IDENT_PORT` | 8351 | |
41. | `IDENT_MAIL_SINK` | — | file that gets `<address> <code>` appended per code INSTEAD of mail. Dev: `storage/mail-sink.txt` |
42. | `SMTP_HOST` `SMTP_PORT` `SMTP_USER` `SMTP_PASSWORD` `SMTP_FROM` | — / 587 | hl:smtp; used when `IDENT_MAIL_SINK` is empty. Neither set → the code is only logged as NOT SENT |
43. | `IDENT_STORAGE` | `./storage/mpackdb` | table DIRECTORY (`<dir>/<table>.*`); absolute for tools |
44. | `IDENT_SESSIONS` | `.sessions/` | session store |
45. | `IDENT_OTP_TTL_MS`, `IDENT_SEND_WINDOW_MS`, `IDENT_SEND_LIMIT` | 600000, 600000, 3 | clocks/limits (the gate shortens the OTP TTL) |
46. | `IDENT_IP_LIMIT`, `IDENT_IP_WINDOW_MS` | 10, 600000 | codes per client IP per 10 min (mission 010) |
47. | `IDENT_IP_DAY_LIMIT`, `IDENT_IP_DAY_WINDOW_MS` | 30, 86400000 | codes per client IP per 24 h |
48. | `HL_HOST` (or `HOST`) | 0.0.0.0 | interface to bind; `127.0.0.1` on Byrodin (hl:web reads it itself, hybriel#24) |
49. | `IDENT_WATCH` | on | `0` = no dev watcher (the container) |
50. | `IDENT_GRANT_TTL_MS` | 60000 | life of a login button's one-time code (`tests/apps.mjs` shortens it) |
52. ## What it does (piece 1)
54. * **Login**: email → 6-digit code (10 min, single use, 5 wrong tries kill it, a new request
55. replaces the old code, max 3 codes per address per 10 min, **max 10 codes per client IP
56. per 10 min and 30 per 24 h**) → signed in.
57. * **Per-IP limit** (mission 010): the code request is `POST /api/code {email}` (the sign-in
58. form fetches it; it is NOT a face any more — a face sees no request headers). The client IP
59. is the **`X-Client-IP`** header only (nginx on Byrodin sets it and overwrites a client's:
60. `/CONTAINERS/web/nginx/conf.d/cloudflare-client-ip.conf`); `CF-Connecting-IP`,
61. `X-Forwarded-For`, `X-Real-IP` are never read. IPv6 counts per **/64**, `::ffff:a.b.c.d`
62. as the IPv4. **No header (dev, no nginx) = ONE shared bucket `direct`** (hl:web's
63. `req.remoteAddress` would only be nginx's). Refusal: 429 `{error}` shown under the form ("too many codes were
64. requested from your network …"); per address: 429 "too many codes were sent to this address …".
65. Table `ipsends` `{ip (bucket), at}`. **Only safe while ident is reachable through nginx
66. alone** (on Byrodin it binds 127.0.0.1) — anyone reaching it directly could send any header. Session cookie
67. `identsid` (manifest `sessionCookie`, not hl:web's default `hlsid`: cookies ignore ports, tickets on :8350 uses `hlsid`).
68. * **The code step is its own page** (ident#20, mission 032; creator: "just make a /code where
69. it checks a pending code"): after `POST /api/code` succeeds the page calls the face
70. `rememberPending(email)` (records `session.data.pendingEmail`; refused unless a code for that
71. address is really waiting — `store.hl codeWaiting`) and goes to **`/code`** (app login:
72. **`/signin/<rid>/code`**). That route (`project.hl codePage`, a function route because a
73. component route cannot redirect) shows Home's code form for the session's pending address
74. while its code is waiting (unused, unexpired, < 5 wrong tries), else **302 back** to `/` resp.
75. `/signin/<rid>` (also when signed in, or a rid that is not 32 hex). So a reload, a second tab,
76. a re-seed keeps the code form. Cleared on: sign-in (`verifyCode`), "Other address" (face
77. `forgetPending`, → the email page), expiry / killed code (read as none). A wrong code keeps it.
78. After sign-in the address bar is set back to `/` resp. `/signin/<rid>` (`history.replaceState`).
79. * **No registration**: the first right code for an address creates the account **and its
80. default identity** (identity name `Default`), then shows that identity's fields with
81. "all of these fields are optional" and **Save** / **Skip**.
82. * **Identities**: fields identity name, nickname, first name, last name — all optional
83. (trimmed, ≤ 60 chars, no control characters). List, new, edit, delete. The last identity
84. cannot be deleted. The **default** identity is the oldest remaining one (badge in the list).
85. The list shows the identity name; without one: nickname, else first + last name, else
86. `Identity <n>`.
87. * **Avatar** (ticket #15, reworked after the creator asked for an upload): an optional `avatar` field
88. on an identity — a picture **uploaded** in the identity form (file chooser, "Remove picture" to
89. clear). `avatar.js` (route `/avatar.js`, included by `components/main.hl`) cuts it to a centred
90. square, scales it to 128×128 in the browser and writes it as a data URL (WebP; JPEG if the browser
91. cannot make WebP and PNG is too big) into the form's hidden `#favatar`; an `input` event hands it to
92. the page (a page script cannot emit, hybriel #31). The server (`store.hl checkAvatarUrl`) accepts
93. only `data:image/(png|jpeg|webp);base64,` with the type's magic bytes at the start, base64
94. characters only, ≤ 60000 characters — no links, no SVG. Stored in the identity record's `avatar`.
95. Shown as a small round image in the identity list and the "choose an identity" list, and as a
96. preview in the form. Old avatars that were http(s) links (first version) are no longer shown and
97. are dropped on the next save. Apps do not get it yet — handed over once ticket #11 (property
98. hand-over at first handshake) is built; until then apps fall back to the name's first letter.
99. **Lesson (hybriel #32/#41)**: no `if` block next to the text inputs of the form (recreates the form,
100. loses focus) — the preview and the remove button are always rendered and toggled with a `hidden`
101. CSS class from a plain reactive field.
102. * **Time zone**: the browser's IANA zone (`Intl…resolvedOptions().timeZone`) is stored at the
103. first login; shown and changeable (text field + "Use this browser's"). The page refuses names
104. the browser does not know; the server checks the shape only (hl:time has no zones, hybriel #11).
105. A later login does NOT overwrite it.
106. * **Apps** (`/apps`, piece 2): any signed-in account registers apps — name + the origins it
107. runs on (`http(s)://host[:port]`, 1–10). Each app gets a public **API key** (`pk_` + 32 hex)
108. and a **secret** (`sk_` + 48 hex) shown **once** (after register / "New secret"; only its
109. sha256 is stored). List, edit (name, origins; the key stays), new secret (old one dies at
110. once), delete (its per-app ids go with it). Only the owner sees/changes an app.
111. * **One public short id per identity** (ident#23, replaces "one id per app"): every identity has
112. a `shortId` — 5 characters like `a68sz`, random, made when the identity is made, kept forever,
113. the same in every app. Alphabet: the digits 2–9 and the letters a–z without i, l, o (31
114. characters, 28.6 million ids; no 0/O, 1/l/I to mix up on the phone), lower case, unique
115. (checked against an index when made; longer only if it ever runs out). Case does not matter when
116. it is typed (`A68SZ` finds `a68sz`). The identity page shows it (selectable). Only the id is
117. public — names, email and avatar stay as the identity sets them. Identities that existed
118. before get theirs at the first start (`store.hl backfillShortIds`, touched from `project.hl`).
119. The first login of an identity in an app still creates a **connection** (its creation time =
120. "when they registered in it", for the per-app page); it holds no id of its own any more.
121. Connections made before ident#23 keep their old 32-hex `appIdentity` only until the app has
122. migrated (below).
123. * **Migration for the apps** (ident#23): `POST /api/migrate-ids {"key","secret","finish"?}` →
124. `200 {"ids":{"<old per-app id>":"<short id>"},"count":n,"finished":bool}` (every connection of
125. that app that still has an old id; an identity deleted since is left out; 401 wrong key/secret).
126. The app's server calls it once and rewrites its stored users. Until then `/api/notify` also
127. accepts the old id. With `"finish":true` the answer is the same and the old ids are dropped
128. afterwards — from then on they are unknown (404) and the map is `{}`. Gate: `tests/migration.mjs`.
129. * **Ids** (mission 009, creator's convention): every table's key is the mpackdb UUID
130. (`@id`, 12 chars like `0mufcrs6sb3v`); accounts, identities and apps are addressed by it
131. (faces take it as a string; an old numeric id finds nothing). "Oldest first" (the default
132. identity, the app list) sorts by the stored `created`, never by key order.
133. * JSON endpoints: `POST /api/code` (the login code, above), `POST /api/exchange`, `POST /api/kinds`, `POST /api/notify` (piece 4), and the selector's `GET /api/selector/identities`,
134. `POST /api/selector/choose` (CORS, piece 3); other `/api/*` answer a JSON 404. The pages'
135. server faces (`components/home.hl`, `components/apps.hl`) do all other writes. Face arguments are checked strictly (unknown field / wrong type
136. → error naming the field) and every face only trusts a real framework session (see Lessons
137. in STATUS.md — a forged trailing argument is refused).
138. * **Removed (mission 005)**: the mission-003 app login (`/login?app=&return=`, `/continue/:rid`,
139. the origin allow list `IDENT_ALLOWED_ORIGINS`, `testclient/`); old files in
140. `.scratch/removed-mission003/`. Piece 2 rebuilt `/login` and `/api/exchange` per the concept.
141. * **Session hardening (ticket #2)**: a signed-in session gets its OWN expiry, stamped at
142. sign-in (`store.hl` `beginSession`) and checked on every use (`accountOfSession`) —
143. independent of the hl:web session file's own rolling idle/maxAge. `IDENT_SESSION_TTL_MS`
144. (default 14 days). An older session (no expiry stamped yet, e.g. from before this ticket
145. or the mission-009 migration) is upgraded on its next use, not force-signed-out.
146. **"Sign out everywhere"** (`/`, account bar) bumps the account's session epoch, which
147. invalidates every session of that account — the caller's own included — on its next check,
148. without touching any other session file. Ticket #19: it also DELETES every session of the
149. account at once (resident ones and files; `project.hl` `dropUserSessions`, reached through
150. `store.hl` `sessionHooks`) and pushes the `signedOutAll` event (audience: connections whose
151. session is the account's) — `components/main.hl` reloads every open page on every device into
152. the signed-out state, no click. Gate: `tests/signoutall.mjs` (:8702, two Chromes). **"Sign out of this app"** (the per-app page,
153. `/inbox/<connection id>`) forgets one app connection; a later login gets the same short id
154. again. Gate: `tests/hardening.mjs`.
156. ## Invites (ticket ident#22)
158. An app invites people to one of ITS projects through ident. CONCEPT.md does not describe it; the
159. ticket is the spec. Code: `invites.hl`, routes in `project.hl`, table `invites` (`apps.hl`).
160. All calls are `POST` with a strict JSON body and the app's **key + secret** (401 otherwise):
161. ```
162. POST /api/invites {key, secret, project, role, return, uses?, days?, email?}
163. 200 {id, url, state:"open", project, role, uses, expires, mailed}
164. POST /api/invites/list {key, secret, project?} → {invites:[{id, project, role, state, uses, used, identities, expires, created}]}
165. POST /api/invites/get {key, secret, id} → {invite}
166. POST /api/invites/revoke {key, secret, id} → {invite} (409 unless it is open)
167. ```
168. * `project` / `role`: the app's own words (1–60 chars). `return`: where the person lands — like the
169. login button, absolute http(s) on one of the app's origins. `uses` (default **1**, up to 1000),
170. `days` (default **7**, up to 90). `state`: `open`, `used` (all uses taken), `expired`, `revoked`.
171. * The **link** (`<ident>/invite/<token>`) is in the create answer ONLY (ident keeps its sha256). The
172. app shows it (copy / share) itself. With `email`, ident also **mails it through its own mailer**
173. (`mail.hl sendInvite`; sink: `<IDENT_MAIL_SINK>.invites`); max 50 invitation mails per app per 24 h
174. (`IDENT_INVITE_MAIL_LIMIT`, 429). The address is not stored and not tied to the login: whoever holds the link joins.
175. * **The person**: `GET /invite/<token>` — used / expired / withdrawn / unknown give an error page
176. (410 / 404) with a plain message; an open one is the login button's own flow: a login request
177. carrying the invite → `/signin/<rid>` ("You are invited to <app>"), email → code if signed out,
178. the identity choice (one click) if signed in. Choosing an identity **accepts** the invite (takes a use;
179. the same identity again takes none) and sends the browser to `<return>?ident_code=<code>&invite=<invite id>`.
180. * **The app's server** exchanges the code (`/api/exchange`, gives the identity id) and asks
181. `/api/invites/get` for the invite: `identities` lists who accepted — check that the exchanged id is in it.
182. Two people racing for the last use: the second is told "already used".
183. * `IDENT_PUBLIC_URL` (compose: https://ident.worldapi.org) = where links point; unset → the request's Host.
184. `IDENT_INVITE_DAY_MS` (gate only) shortens a "day". Gate: `tests/invites.mjs` (:8700/:8701). The test app has `/invites`.
186. ## How apps use ident (login button, piece 2)
188. 1. **Register the app** in ident: `/apps` → "Register an app": name + origin(s). Keep the API
189. key (public) and the secret (server only; shown once).
190. 2. **The button** sends the browser to
191. `<ident>/login?key=<API key>&return=<percent-encoded return URL>`.
192. The return URL must be absolute http(s), no `#`, ≤ 2000 chars, and its origin one of the
193. app's origins — else ident shows an **error page (HTTP 400) and never redirects**.
194. 3. ident parks the request (30 min) and shows `/signin/<rid>`: signs the browser in to ident
195. if needed (email code; a first login shows the optional names first), then **"Choose an
196. identity"** (also with one identity: it is shown, one click).
197. 4. ident redirects to `<return URL>?ident_code=<code>` (or `&ident_code=` if it has a query).
198. The code is 48 hex, **single use, 60 s**, for this app only.
199. 5. **The app's server** exchanges it:
200. ```
201. POST <ident>/api/exchange {"key":"pk_…","secret":"sk_…","code":"…"}
202. 200 {"identity":"a68sz"} ← the identity's public short id, NOTHING else
203. 401 unknown key or wrong secret 400 unknown/used/expired code, code of another app
204. 400 invalid JSON, unknown/missing/wrong-type field (named); 405 not POST
205. ```
206. A code presented by the wrong app is spent. The same identity always gets the same id, in
207. every app (ident#23); a person can say it aloud and others find them by it. The app asks the
208. user itself for any further data it needs.
210. ### The identity selector (piece 3)
212. Same app registration (key + secret, origins). The app's page includes ident's script and
213. the element, configured with the app's **public API key**:
214. ```html
215. <script src="https://<ident>/selector.js"></script>
216. <ident-selector key="pk_…"></ident-selector> <!-- add logged-in when the site's session is logged in -->
217. <script>
218. const sel = document.querySelector('ident-selector');
219. sel.addEventListener('ident-login', async (e) => { // the user chose an identity = the login
220. await fetch('/my-login', { method: 'POST', body: JSON.stringify({ code: e.detail.code }) });
221. // → YOUR SERVER: POST <ident>/api/exchange {key, secret, code} → {identity}, keep it server side
222. sel.loggedIn = true; // the host tells the selector (or the attribute)
223. });
224. // logout on the website = reset the selector: sel.loggedIn = false / sel.reset()
225. </script>
226. ```
227. * The element (shadow DOM, ident's design) first says **"choose ident"**. Opening it fetches
228. `GET <ident>/api/selector/identities?key=` **with credentials** (the browser's ident
229. session cookie; same site only — `SameSite=Lax`). ident answers **only** when the
230. request's `Origin` is one of the app's registered origins **and** the key is that app's:
231. then `Access-Control-Allow-Origin: <that origin>` + `…-Allow-Credentials: true` (never
232. `*`) and `{"signedIn":true,"identities":[{"id","name"}]}` — the identity **name** only;
233. `id` is an opaque per-app pick id (not ident's id, not the short id). Otherwise
234. **403 without CORS headers** (the page gets nothing). Signed out of ident:
235. `{"signedIn":false,"identities":[]}` → the selector says so and links to ident (new tab);
236. opening it again re-asks.
237. * Choosing an identity: `POST <ident>/api/selector/choose?key=` `{"identity":"<id>"}`
238. (preflighted, same CORS rule; strict body) → `{"code"}` — **the same one-time code as the
239. login button's** (48 hex, single use, 60 s, this app only). The element fires
240. **`ident-login`** (`event.detail.code`, bubbles, composed) and closes. The host's server
241. exchanges the code exactly as in step 5 above.
242. * **`logged-in`** attribute / `loggedIn` property: set by the HOST (it knows its own
243. session): the element shows "✓ logged in with ident" (+ the identity's name if chosen on
244. this page) instead of the button. Removing it (`loggedIn = false`, `reset()`) = back to
245. "choose ident". The selector sets **no cookie** and keeps no login state of its own.
246. * **Restyle** from the host page: CSS custom properties on the element —
247. `--ident-accent`, `--ident-accent-text`, `--ident-background`, `--ident-text`,
248. `--ident-text-strong`, `--ident-text-muted`, `--ident-border`, `--ident-radius`,
249. `--ident-font` — and `::part(button | panel | identity | status | message | link)`,
250. e.g. `ident-selector { --ident-accent: #569bd4 } ident-selector::part(button) { text-transform: uppercase }`.
252. **Try it** (test app, `testapp/`, a separate hybriel app): http://100.77.141.84:8354 —
253. register an app in ident with origin `http://100.77.141.84:8354`, paste key + secret into
254. the test app's form, press "Log in with ident" — or, on the same page, **"choose ident"** (the selector; you
255. must be signed in to ident at http://100.77.141.84:8351 in the same browser). The test app
256. shows the id it got (a real app never would) and "new user"/"welcome back"; the selector
257. switches it to "Logged in" without a reload, "Log out" resets the selector. Its own session
258. cookie: `testapp<port>sid`. Its data: `testapp/storage/testapp.json`.
259. ```bash
260. cd testapp && TESTAPP_PORT=8354 TESTAPP_URL=http://100.77.141.84:8354 IDENT_URL=http://100.77.141.84:8351 \
261. setsid nohup ../bin/hybriel project.hl > testapp.log 2>&1 < /dev/null & echo $! > testapp.pid
262. ```
264. ## How apps send notifications (piece 4)
266. CONCEPT.md "Notifications" / "Per-app page". The app's SERVER calls ident with its API key +
267. **secret** (never from a page) and the **app-specific identity id** it got from the exchange.
268. Apps never get an email address. Strict JSON bodies (unknown/missing/wrong-type field → 400
269. naming it; invalid JSON 400; not POST 405; wrong secret / unknown key 401).
271. 1. **Register the kinds** (by name) with their **preset channels** — optional, but a kind must
272. be registered before it can be **pushed**. Upsert: a name already there gets the new
273. preset. Answers every registered kind. `kinds: []` just lists them. ≤ 50 per call, ≤ 200 per app;
274. a refused call writes nothing.
275. ```
276. POST <ident>/api/kinds {"key":"pk_…","secret":"sk_…","kinds":[{"name":"New comment","push":true,"email":false}]}
277. 200 {"kinds":[{"name":"New comment","push":true,"email":false}]}
278. ```
279. 2. **Send** a notification: `name` (1–60 chars, one line — the kind), `text` (1–2000 chars,
280. Markdown by the worldapi convention; the inbox shows it as plain text with its line breaks),
281. optional `icon` and `link` (absolute http(s) URLs ≤ 2000), optional `urgent` (bool).
282. ```
283. POST <ident>/api/notify {"key":"pk_…","secret":"sk_…","identity":"a68sz","name":"New comment",
284. "text":"Bea commented …","icon":"https://…/i.png","link":"https://…/post/7","urgent":false}
285. 200 {"id":"<notification id>"}
286. 404 unknown identity for this app (the identity never logged in to this app, an unknown id;
287. until the app has migrated, its old per-app ids work too)
288. ```
289. **An unregistered name is allowed**: it becomes an unregistered kind with the concept's
290. default (daily mail, no push); push stays impossible for it until the app registers it.
292. **Effective settings** (per connection = identity in app, per kind; stored in `settings`):
293. the app's preset, replaced by the user's switch where the user set one; the user's **override
294. for all**, when on, replaces both for every kind; push is always off for an unregistered kind.
295. **Channels of a notification**, fixed when it arrives (stored on it, with `pushSent`/`mailSent`
296. = false — delivery is pieces 5/6): normal → `push` = effective push, `mail` = `daily` if
297. effective email else `none`; urgent → push if effective push AND a push device exists (none
298. before piece 6), else `mail = now` if push or email is on for it, else `none`. Every
299. notification lands in the inbox.
301. **The user**: `/inbox` (nav "Inbox") = all notifications of all identities, newest first
302. (max 200 shown): name, app · identity, text, icon (`referrerpolicy=no-referrer`), "Open"
303. (the action link, new tab), time (UTC — hl:time has no zones), urgent badge, read/unread
304. toggle. Below: every app connection (app, identity, since) → **per-app page**
305. `/inbox/<connection id>`: app, identity, "registered in this app since" (the connection's
306. `created`), **All notifications** (Override all / Push / Email switches) and every kind of the
307. app with Push and Email switches (effective state; "app default: …" under the name; an
308. unregistered kind shows "—" for push; locked while the override is on). Each flip is saved at
309. once. Faces: `inboxMark`, `inboxSwitch`, `inboxOverride` (session-checked, #31).
311. **Try it** (dev): the test app shows the identity id it got; with the app's key + secret:
312. ```bash
313. curl -s -X POST http://100.77.141.84:8351/api/notify -H 'content-type: application/json' \
314. -d '{"key":"pk_…","secret":"sk_…","identity":"<id>","name":"Hello","text":"First notification"}'
315. ```
316. then open http://100.77.141.84:8351/inbox.
318. ## Test
320. ```bash
321. node tests/browser.mjs # THE GATE: 132 checks, ~35 s (ident#20 /code: 38 of them). Own servers :8356 (ident) and :8357
322. # (OTP TTL 3 s, reached as localhost); own storage .scratch/gate-store;
323. # two Chromes on debug ports 8640-8659 (--disable-gpu), time zones
324. # emulated (Pacific/Auckland, Asia/Tokyo). Covers avatar: save/edit,
325. # live preview, list display, URL validation negatives (ticket #15)
326. node tests/apps.mjs # PIECE 2 GATE: 105 checks, ~15 s (ident#20 /signin/<rid>/code: 8). Own servers: ident :8370, ident :8371
327. # (code TTL 1.5 s), test apps :8372/:8373; storage .scratch/apps-gate;
328. # two Chromes on 8670-8679. Screenshots .scratch/apps-*.png
329. node tests/selector.mjs # PIECE 3 GATE: 99 checks, ~10 s. Own servers: ident :8390, test apps
330. # :8391/:8392, a node page on :8393 (an unregistered origin);
331. # storage .scratch/selector-gate; Chromes on 8690-8699.
332. # Screenshots .scratch/selector-*.png
333. node tests/shortid.mjs # SHORT ID GATE (ident#23): 19 checks, ~10 s, HTTP only. Own ident :8706 (storage
334. # .scratch/shortid-gate): every identity's id, unique, same in two apps, notify, migrate-ids
335. node tests/migration.mjs # MIGRATION GATE (mission 009 + ident#23): 33 checks, ~10 s. Old-format fixture
336. # (tests/oldstore-009.hl) → tools/migrate-009.hl twice on a copy →
337. # ident :8395 + test app :8396 on the result: old session cookies
338. # still signed in, stored per-app ids come out of the exchange again
339. # (Chrome, login button → "welcome back"), pre-migration code still
340. # works. Storage .scratch/migration-gate; Chrome on 8710-8719
341. node tests/notify.mjs # PIECE 4 GATE (mission 013): 130 checks, ~20 s. Own ident :8410 +
342. # a node "app site" :8413 (icon + link target); storage
343. # .scratch/notify-gate (the stored state is read off COPIES with
344. # tools/dump-store.hl); Chromes on 8740-8749.
345. # Screenshots .scratch/notify-{inbox,inbox-read,appsettings,override}-*.png
346. node tests/iplimit.mjs # PER-IP GATE (mission 010): 39 checks, ~15 s. Own servers: ident :8400
347. # (HL_HOST=127.0.0.1, IP limit 3), :8401 (IP window 2 s, day limit 5);
348. # storage .scratch/iplimit-gate; Chromes on 8720-8739, each with its
349. # own X-Client-IP (CDP). Screenshots .scratch/iplimit-refused-*.png
350. # the other gates run their servers with IDENT_IP_LIMIT/IDENT_IP_DAY_LIMIT=1000 (no header =
351. # one shared bucket, and they request many codes)
352. node tests/hardening.mjs # ticket #2: 17 checks. node tests/signoutall.mjs # ticket #19: 6 checks
353. # tests/dev-smoke.mjs: DO NOT RUN any more — the dev server sends real mail now
354. ps -eo pid,args | grep [h]l-browser-tier # must print nothing afterwards
355. ```
357. The gate covers, in real browsers: signed-out login page, first login → account + default
358. identity + optional-names form, **Skip** (A) and **names filled** (B), reload keeps the
359. session, edit, cancel, second and third (empty) identity, delete with a dismissed and an
360. accepted confirm, deleting the default one, the last one cannot be deleted, time zone from
361. the browser at first use / change / unknown zone refused / "Use this browser's" / not reset by
362. a later login, sign out; OTP rules (lowercasing, wrong-code countdown, 5 tries, replaced code,
363. rate limit, single use, expiry on the short-clock server); face negatives over
364. `POST /__hl/emit` (no session, unknown/wrong-type/too-long/control-char fields, another
365. account's identity, forged session argument); the removed routes are 404; palette; no console
366. errors; 390px/1280px without horizontal overflow. Screenshots: `.scratch/gate-*.png` — look.
367. **ident#20 (mission 032), the code page `/code`**: "Send me a code" lands on `/code`; GET `/code`
368. without a pending code / without a cookie / signed in → 302 `/`; SSR of `/code` (plain GET with
369. the cookie) = the code form + address; a real `Page.reload` keeps the code form (390/1280 px,
370. `.scratch/gate-*-code-reloaded.png`); a second tab of the same browser shows it too; a wrong code
371. + reload keeps it; one mail only; the code typed after the reloads signs in (welcome, URL `/`);
372. signed out → `/code` lands on the email form; "Other address" → `/`, server forgot it, reload =
373. email form; 5 wrong codes → reload lands on `/`; face `rememberPending` refuses an address without
374. a waiting code, a number, a forged trailing session, accepts (normalised) an address with a waiting
375. code — for that session only; `forgetPending`; `/signin/xyz/code` and a CR/LF rid → 302 `/`;
376. `/signin/<rid>/code` with nothing pending → 302 `/signin/<rid>`; short-clock server: reload right
377. after asking = code form, expired code → reload lands on `/`, expiry without a try → `/`.
378. `tests/apps.mjs` adds (app login): "Send me a code" → `/signin/<rid>/code`, reload keeps the code
379. form under the app banner (`.scratch/apps-code-for-app-reloaded-*.png`), the code signs in (URL back
380. to `/signin/<rid>`) and the app flow completes; "Other address" → `/signin/<rid>` email form, a
381. reload stays there, `/signin/<rid>/code` with nothing pending → `/signin/<rid>`.
382. The other gates' login helpers wait for the `/code` page to hydrate before typing the code.
383. (Ticket #37, mission 007: its two checks of the removed mission-003 routes now check that
384. the old shapes are refused by the piece-2 routes: `POST /api/exchange {"code"}` → 400
385. `missing field: key`; `/login?app=…` → 400 error page, no redirect. 81 passed.)
387. `tests/apps.mjs` covers, in real browsers: /apps signed out; register (bad origin refused,
388. secret shown once, not after reload), list, edit; test app setup; login button signed in
389. with one and with two identities (same identity + app → same id, other app → other id,
390. other identity → other id, the answer is only `{identity}`); signed out → ident login →
391. (first login: optional names, then) choice → back; login negatives (foreign origin, the
392. other app's origin, `user@host` trick, `javascript:`, missing/unknown key, unknown request
393. id); exchange negatives (wrong secret/unknown key 401, reused, other app's code, expired,
394. unknown, invalid JSON, not an object, missing/unknown/wrong-type field, surrogate escape,
395. GET); forged session argument on every new face (#31), strict app fields, another account
396. cannot touch an app or use an identity; new secret (dismissed/accepted confirm, old secret
397. 401, the test app with the old secret fails visibly), delete (its key → error page);
398. no console errors; 390/1280 px without overflow.
400. `tests/selector.mjs` covers, in real browsers: signed in to ident on ident's origin → the
401. test app's page (own origin) shows the selector ("choose ident", ident's colours), open →
402. the identities by name, the answer holds only `{id, name}` (no email, opaque ids) → choose →
403. `ident-login` code → host exchange → logged in WITHOUT reload (a window marker survives),
404. `logged-in` set, status shows the identity, no cookie from the selector → reload: the
405. host renders `logged-in` → host logout resets (no reload) → other identity → other id; the
406. button flow gives the same id as the selector; app B → other ids and other pick ids;
407. host restyle (custom properties and `::part`) changes computed styles; negatives:
408. unregistered origin (same site, so the cookie travels — the Origin check alone stops it; the
409. page's own fetch is CORS-blocked, the preflight too), another app's key / an unknown key,
410. signed out of ident (second browser, and after sign-out), 403 without CORS headers for
411. bad/missing/`null`/prefix origins, no key, made-up cookie, preflight good/bad, strict choose
412. body (unknown/missing/wrong-type field, invalid JSON, a forged `session` field), ident's own
413. identity id or another app's pick refused, another account cannot choose alice's identity,
414. code reuse / other app's code / spent code, 405s, forged face sessions on 8 faces (#31);
415. console clean; 390/1280 px without overflow, closed and open.
417. `tests/notify.mjs` covers: the app registers kinds (strict body, 401s, refused call writes
418. nothing, re-register changes presets), sends normal/urgent, with/without icon + link, a
419. registered and an unregistered name, to two identities in two apps (+ another account);
420. send negatives (wrong secret, unknown key, another app's id, ident's own ids, unknown/missing/
421. wrong-type fields, bad URLs, control characters, surrogate escape, GET); stored channels per
422. notification (push/daily/now/none, not delivered); in Chrome: inbox (all identities, newest
423. first, app · identity, text line breaks, icon loaded, link, urgent badge, time, unread count),
424. mark read / unread (survives reload, stored); per-app page (since = connection created, all
425. kinds, presets, unregistered = no push), flip switches → stored effective settings change,
426. survive reload, a later notification follows them; override on (locked switches, all kinds
427. follow, user switches kept) and off again; forged/no session on the 3 faces (#31), bob can't
428. see/mark/switch alice's, alice can't switch bob's, unknown connection, signed out; bob's own
429. Chrome sees only his; console clean; 390/1280 px without overflow.
431. `tests/iplimit.mjs` covers: two IPs via `X-Client-IP` (the limit hits one, not the other);
432. `X-Forwarded-For` / `CF-Connecting-IP` / `X-Real-IP` / `Forwarded` beside it change nothing;
433. lower case / blanks; no header = one shared bucket (forwarded headers do not split it), empty
434. header = none; IPv6 /64 buckets, IPv4-mapped; per-address limit across IPs; refused codes
435. are not mailed; the old `requestCode` face is gone; 405 / strict body / invalid JSON; the
436. short window expires, the day limit holds; `HL_HOST=127.0.0.1` not reachable on the LAN
437. address, without it 0.0.0.0 as before; two real Chromes (one IP each): A's 4th code refused
438. VISIBLY on the page (390/1280 px, no overflow), B signs in, reload keeps the session.
440. ## Deploy (Byrodin)
442. Target: `/CONTAINERS/projects/ident.worldapi.org` on Byrodin, container `ident.worldapi.org`
443. (`docker-compose.yml`: debian:12-slim, host network, `HL_HOST=127.0.0.1`, `IDENT_PORT=45002`,
444. `IDENT_WATCH=0`, the folder mounted at `/home/ident`, `./bin/hybriel project.hl`), public
445. https://ident.worldapi.org/ via nginx (TLS ends there; no baseUrl/tls in the app, like notes).
446. * **First deploy: done by the architect** (folder, `.env` with SMTP on Byrodin, nginx vhost with
447. WebSocket Upgrade headers and `proxy_set_header X-Client-IP $client_ip;`, cert, DNS).
448. * **Later: `./deploy.sh`** on Loreana, in this folder: runs the six gates (refuses on a
449. failure; `--skip-tests` skips them LOUDLY), rsyncs the code to
450. `[email protected]:/CONTAINERS/projects/ident.worldapi.org` (never `storage/`, `.sessions/`,
451. `.env`, `.scratch/`, `server.*`, `testapp/`, logs — the preview is checked for them; no
452. `--delete`), `docker compose up -d && docker compose restart` over `ssh -F /dev/null`, then
453. waits for https://ident.worldapi.org/ to answer 200. Every step is printed.
454. * `./deploy.sh --dry-run` = gates + `rsync -n` + the commands it would run (no restart, no URL
455. check). `--target DIR|HOST:DIR` and `--url URL` point it elsewhere (tested only against a
456. local directory: see STATUS "mission 010").
457. * Session cookie `identsid`: `HttpOnly; SameSite=Lax`, no `Secure` (hl:web sets it — checked 2026-09-26: `identsid=…; Path=/; HttpOnly; SameSite=Lax; Max-Age=1209600`; it works
458. on https — checked behind a local TLS proxy). The selector's CORS is exact-origin: register
459. apps with their https origin (`https://tickets.worldapi.org`); the `http://` twin is refused.
461. ## Data
463. `storage/mpackdb/{accounts,identities,otp,sends,ipsends,apps,connections,requests,grants,kinds,settings,notifications}.*`
464. (hl:mpackdb, pk `@id` UUID; no admin UI/API). Tables and fields: the headers of `store.hl`
465. and `apps.hl`, `notify.hl` (piece 4: `kinds`, `settings`, `notifications` — new tables, created
466. empty at the first start; no migration). **Opening a table rewrites its files (hybriel #40): only ever open a COPY.**
467. Everything as JSON (on a copy; the output holds apps' secret HASHES — keep it in .scratch):
468. ```bash
469. rm -rf /tmp/identcopy && cp -a storage/mpackdb /tmp/identcopy
470. IDENT_STORAGE=/tmp/identcopy ./bin/hybriel tools/dump-store.hl | python3 -m json.tool | less
471. ```
472. Quick look without opening: `strings storage/mpackdb/accounts.mpack`. Dev mail:
473. `storage/mail-sink.txt`. Mission-003 data: `.scratch/storage-backup-20260924-060627/`.
475. ### Migration 009 (`*id` → UUID, done 2026-09-24)
476. The old tables `storage/ident-<table>.*` (pk `*id`, public ids = id + 1) stay in `storage/`
477. as a backup; nothing reads them. `tools/migrate-009.hl` (one-off, idempotent: rows carry
478. `oldId` = the old public id; a second run writes nothing) copied every row of every table,
479. rewrote the references to the new UUIDs, kept `apiKey`, `secretHash`, `appIdentity`, `rid`,
480. grant hashes byte-identical, and rewrote the session files' `"user":{"id":<n>}` to the
481. account's UUID (the creator stayed signed in). Run it only on a COPY of the old tables with
482. the server stopped (header of the tool). A connection of an identity deleted before the
483. migration keeps its per-app id with `identity = 'gone:<old id>'`.
484. Selector pick ids (sha256(secret hash : identity id)) changed once with the identity ids —
485. harmless, the selector re-fetches them on every open.
487. ## Files
489. | File | |
490. |---|---|
491. | `CONCEPT.md` | the creator's concept — do not edit |
492. | `project.hl` | manifest: routes (`/`, `/signin/:rid`, `/apps`, `/inbox`, `/inbox/:cid`, `/login`, `/api/code`, `/api/exchange`, `/api/kinds`, `/api/notify`, `/api/selector/*`, `/selector.js`, `/api/*` 404, `/code` + `/signin/:rid/code` → `codePage`), cookie name, the listener (`HL_HOST`) |
493. | `apps.hl` | tables apps / connections / requests / grants and the login button + exchange rules (`issueCode`: the one-time code of both flows) |
494. | `selector.hl` | the selector's two CORS endpoints (origin + key check, pick ids, choose → code) |
495. | `selector.js` | the browser half: `<ident-selector>` custom element (shadow DOM), served at `/selector.js` |
496. | `store.hl` | tables accounts / identities / otp / sends / ipsends and all rules (codes stored as sha256 only; `ipBucket`, the per-IP limit; the pending sign-in `codeWaiting`/`recordPending`/`pendingOf`/`dropPending`, ident#20) |
497. | `components/home.hl` | `/` and `/signin/:rid` (and, rendered by `codePage` with `step = 'code'`, `/code` + `/signin/:rid/code`): login, welcome, identities, time zone, the identity choice for an app |
498. | `components/apps.hl` | `/apps`: register / list / edit / new secret / delete apps |
499. | `components/main.hl` | shell (header) |
500. | `api.hl` | JSON replies, redirect, strict JSON body, the login button's error page |
501. | `jsoncheck.hl` | JSON syntax pre-check (copied from tickets, hybriel #12) |
502. | `testapp/` | the minimal app using the login button and the selector (:8354) |
503. | `mail.hl` | hl:smtp Mailer + `IDENT_MAIL_SINK` |
504. | `styles.hl` | all CSS (imports the tokens from `shared/tokens.hl`; accent `colorAccent = var(orange)` = #ce9178, danger `colorDanger` = `var(--red)` = #f44747) |
505. | `shared/tokens.hl` | the WorldAPI tokens (hl:web `var()`), the hl:web copy all apps share (see "Design tokens"); README "Design tokens" |
506. | `tests/browser.mjs`, `tests/apps.mjs`, `tests/selector.mjs` | gates of pieces 1, 2, 3; `dev-smoke.mjs` retired (real mail); `cdp.mjs`/`ports.mjs` copied from tickets |
507. | `tests/migration.mjs`, `tests/oldstore-009.hl` | the migration gate and its old-format fixture (mission 009; also the short-id migration, ident#23) |
508. | `tests/shortid.mjs` | the short id gate (ident#23) |
509. | `tools/migrate-009.hl` | one-off migration `*id` → UUID + sessions (done; kept for the record) |
510. | `tools/dump-store.hl` | every table as JSON — on a COPY only |
511. | `tests/iplimit.mjs` | the per-IP limit gate (mission 010) |
512. | `notify.hl` | piece 4: tables kinds / settings / notifications, the send + kinds rules, effective settings, inbox and per-app page data |
513. | `components/inbox.hl` | `/inbox`: the notifications of all identities, read/unread, the app connections |
514. | `components/appsettings.hl` | `/inbox/:cid`: the per-app page (since, override, per-kind push/email switches) |
515. | `tests/notify.mjs` | the piece-4 gate (mission 013) |
516. | `tests/signoutall.mjs` | ticket #19 gate: sign out everywhere pushes to open pages on other devices, session files deleted (:8702, Chrome debug 8703-8709) |
517. | `tests/hardening.mjs` | ticket #2 gate: session own-expiry, sign out everywhere, sign out of an app (own servers :8700/:8701) |
518. | `docker-compose.yml`, `deploy.sh` | Byrodin container; the deploy from Loreana (section "Deploy") |
520. ## Design tokens (shared, ticket antcolony#3 — mission 021)
522. * **`shared/tokens.hl`** declares the WorldAPI palette + semantic tokens as hl:web css variables,
523. hand-written: `static dark = var('rgb(25, 30, 35)')`, `static colorText = var(light)`, …
524. (`import { var } from 'hl:web/css'`). It is a copy of the one source
525. `loreana:/media/STORAGE/projects/worldapi-tokens/tokens.hl` (vendored like `plugins/`, no generator):
526. edit it THERE, then `cp /media/STORAGE/projects/worldapi-tokens/tokens.hl shared/tokens.hl` in every
527. app and restart it. Check: `cmp /media/STORAGE/projects/worldapi-tokens/tokens.hl shared/tokens.hl`.
528. (2026-09-26: the apps' copy = gitoria's; it differs from the source only in 3 COMMENT lines that
529. still say webex — update the source's comments, then the check is byte-exact again.)
530. * `styles.hl` IMPORTS the tokens it uses (`import { colorText, colorBorder, … } from './shared/tokens.hl'`)
531. and writes them as members: `color = colorText`, `border = '1px solid ' + colorBorder`. It sets only
532. its accent: `colorAccent = var(orange)` (#ce9178). A token it uses must be in the import list,
533. a new token must be added to tokens.hl (static) first.
534. * hl:web names each token after its member (`colorTextMuted` → `--color-text-muted`), writes EVERY
535. token of tokens.hl into the one `:root` (declaration order), then the app's `colorAccent` (a second
536. `--color-accent`, later wins), and writes each use as `var(--…)`. Components/JS may still use
537. `var(--color-…)` strings — the custom property names are the same.
538. * hl:web does this itself since hybriel#39 (the webex LOCAL PATCH of mission 021 is gone, mission 036).
539. * A change of tokens.hl or of styles.hl root members needs a RESTART: the dev watcher re-analyses
540. but the served sheet keeps the old values (measured, mission 021).
541. * Deploy: `shared/` is part of the app folder; `deploy.sh`'s rsync sends it (proved in mission 021:
542. deploy.sh excludes + debian:12-slim container → byte-identical `/__hl/app.css`).
544. ## Vendored Hybriel
546. **hybriel master 837fe120** (mission 036, 2026-09-26; includes #103/#104 = d9aa12e7..0c285350).
547. `bin/hybriel` sha256 `9e5e95b33680eb68a016e9e48a76c9f92193fd2ffdbdf437c1aab1bda2731a0d`, built read-only
548. (`git -C /media/STORAGE/projects/hybriel archive master | tar -x -C ~/scratch-036/src`, then
549. `cd native && /media/STORAGE/projects/termuplex/.tools/zig/zig build -Dtarget=x86_64-linux-gnu.2.39 -Doptimize=ReleaseFast`).
550. `plugins/` = master's core crypto data fetch fs http http1 mpackdb proc smtp time web — **no local patch**.
551. Re-vendor = copy the binary + these plugins, run every gate. Old webex generation (e565176b + LOCAL PATCHes
552. #34 seed escape, #39 tokens) backed up in `.scratch/pre-036/` (bin, plugins, sources, tests).
554. * Framework pages use content-hashed URLs: `/__hl/hl-runtime.js?v=…`, `/__hl/web/client.js?v=…`,
555. `/__hl/app.css?v=…` (`Cache-Control: public, max-age=31536000, immutable`; bare `/__hl/app.css` = `no-cache`).
556. Check: `~/scratch-036/hashcheck.sh ident.worldapi.org IDENT /` (own server :8730, temp storage).
557. * **A forged trailing session argument is refused by hl:web itself** (hybriel#16): the ack is
558. `ok:false`, "… the `session` parameter is filled by the server, never by the peer" — the face never
559. runs. The gates' forged-session checks (20: browser 4, apps 5, selector 8, notify 3) accept that
560. refusal (`framework_refused`) or the app's own `{error}`. `realSession()` is gone; faces check `session == null`.
561. * Kept on purpose: `mail.hl` placeholder host 127.0.0.1 (a Mailer must exist for `deliver()`/handlers),
562. `jsoncheck.hl` (JSON.parse still aborts on bad input), `X-Client-IP` for the IP limit (behind nginx
563. `req.remoteAddress` is nginx), hand sorts (no list `sort()`), `avatar.js` guard.