diff --git a/AGENTS.md b/AGENTS.md index 1ecb963e8..7c1d62f9b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -8,7 +8,7 @@ repo. If a fact here conflicts with source files or CI config, trust the source Memos is a self-hosted note-taking app. - Backend: Go 1.27.0, Echo v5, Connect RPC, gRPC-Gateway, Protocol Buffers. -- Frontend: React 19, TypeScript 6, Vite 8, Tailwind CSS v4, React Query v5. +- Frontend: React 19, TypeScript 7, Vite 8, Tailwind CSS v4, React Query v5. - Storage: SQLite, MySQL, PostgreSQL. - Generated API outputs: `proto/gen/` for Go/OpenAPI, `web/src/types/proto/` for TypeScript. @@ -21,6 +21,13 @@ Memos is a self-hosted note-taking app. - Add public API endpoints to `server/api/v1/acl_config.go`. - Ask before adding heavy dependencies, changing auth/token behavior, or altering Docker/release workflows. +## Documentation + +- ADRs and their companion glossary have been retired. Do not recreate them unless explicitly requested. +- Design documents in `docs/design/` are frozen after implementation. Do not update implemented designs to track later code changes. +- New and unimplemented designs may still evolve. Frozen documents record the original design; verify current behavior in source code and tests. +- Keep contributor guidance in this file and package-local `README.md` or `doc.go` files current when relevant code changes. + ## Commands Run from the repository root unless a command starts with `cd`. @@ -64,7 +71,7 @@ Every root folder is named for what it holds. Read the folder's `doc.go` or `REA | `server/frontend/` | Static SPA serving; `dist/` is the built SPA baked in by `go:embed` | | | `server/mcp/` | Model Context Protocol server | | | `server/auth/` | JWT access tokens, refresh tokens, PAT handling | | -| `core/` | Business rules with no HTTP and no SQL: `access`, `notification`, `memopayload` | store, provider, markdown, filter, internal | +| `core/` | Business rules with no HTTP and no SQL: `access`, `notification`, `memopayload`, `memoexport` | store, provider, markdown, filter, internal | | `store/` | Store facade, cache, migrations, `Driver` interface; `store/db/{sqlite,mysql,postgres}/` implement it | provider, markdown, filter, internal | | `markdown/` | Markdown engine: parser, AST, memos syntax extensions, renderer, memo payload | proto/gen, internal | | `filter/` | CEL filter compiler: parse to IR, render to SQL per driver, filterable field schema | internal | @@ -112,7 +119,7 @@ Black-box service tests live in `server/api/v1/test/`, unit tests next to the co - Follow Biome formatting: 2-space indent, double quotes, semicolons, 140-character line width. - Put server data in React Query hooks under `web/src/hooks/`; keep UI-only state in contexts or component state. - Use Tailwind CSS v4 utilities, `cn()` for class merging, and CVA for variants. -- Reuse Radix primitives and existing components before adding new UI primitives. +- Reuse Base UI primitives and existing components before adding new UI primitives. - Keep generated proto TypeScript under `web/src/types/proto/` out of manual edits and Biome rewrites. ## Database And Proto Rules diff --git a/docs/adr/0001-tag-syntax-and-recognition.md b/docs/adr/0001-tag-syntax-and-recognition.md deleted file mode 100644 index 792489a08..000000000 --- a/docs/adr/0001-tag-syntax-and-recognition.md +++ /dev/null @@ -1,635 +0,0 @@ -# ADR 0001: Tag Syntax and Recognition - -Status: Accepted - -Date: 2026-08-01 - -Domain glossary: [Memos domain glossary](../glossary.md) - -## Context - -Tags are part of Memos' Markdown language, not merely an editor decoration. A tag occurrence affects rendered memo content, the derived tags stored in a -memo payload, API responses, tag counts, filters, tag metadata, navigation, and editor completion. - -Memos currently has four related but different implementations: - -- The Go Markdown parser recognizes Unicode letters, marks, numbers, and symbols, plus `ZWJ`, `_`, `-`, `/`, and `&`. It extracts at most 100 Unicode - code points and has no general left boundary. -- The read-only remark plugin uses a similar frontend character class without `ZWJ`. It rejects a run longer than 100 code points and has special handling - for adjacent `#` characters. -- The editor decoration uses the frontend character class but requires a non-letter/non-number before `#`, so it does not highlight `hello#tag`. It scans - raw text without consulting the editor's Markdown syntax tree. -- Editor completion has no left boundary, no length limit, and no Markdown-context check. - -These differences expose implementation drift but do not define the desired language. This ADR starts from intended product semantics; current behavior -is non-normative. - -This ADR defines the current Memos tag syntax and recognition rules, covering: - -- The lexical form of a tag identifier. -- How a tag candidate is found in flowing Markdown text. -- Which Markdown contexts can contain a tag occurrence. -- Pinned Unicode and emoji data. - -Existing-data migration, rollout, backfill, and metadata remapping are outside this ADR's scope. - -Markdown context is defined against the [GFM 0.29-gfm specification](https://github.github.com/gfm/). Memos-specific Markdown extensions layer on that -baseline and are opaque to tag recognition unless their own definition explicitly exposes ordinary text. - -The slash is a hierarchy separator between non-empty segments. `-`, `+`, and `&` are unrestricted visible segment units. Tag equality is exact: the display -value itself is the comparison key, with no case folding or Unicode normalization. - -A tag is a value derived from memo Markdown, not a durable entity with an identity independent of its source occurrences. Memos cannot rename a tag by -changing metadata or a separate tag record. Renaming `#Work` to `#work` across 100 memos requires editing those 100 memo sources. - -## Decision drivers - -- Work naturally for multilingual personal notes, including numeric tags and emoji. -- Preserve common word-internal apostrophes without absorbing surrounding quotation punctuation into tag values. -- Recognize `hello#tag` intentionally without mistaking URL fragments for tags. -- Give the backend, renderer, editor decoration, and completion the same values and source spans. -- Use Markdown syntax rather than ad hoc URL or code regular expressions. -- Avoid accepting every Unicode symbol merely to support emoji. -- Keep parsing deterministic across Go, JavaScript runtimes, and Unicode upgrades. -- Keep lexical scanning linear in memo size. - -## Terminology - -**Tag** -: A classification value in a memo tag set, derived from one or more tag occurrences either as a direct tag value or as an implied ancestor. A tag is not - independently created, owned, or renamed outside the source text that produces it. - -**Introducer** -: An ASCII number sign `#` (U+0023) that is not part of a matched fully-qualified emoji sequence and begins a tag candidate. It is not included in the - extracted tag value. - -**Tag identifier** -: The non-empty Unicode code-point sequence emitted from a tag source spelling, for example `work/notes` from `#work/notes`. - -**Tag candidate** -: An introducer followed by a source spelling that matches the lexical grammar entirely within one literal-source run, before that run's Markdown context - is checked for eligibility. - -**Tag occurrence** -: A tag candidate whose complete recognized source span is in an eligible literal-source run. - -**Literal-source run** -: A contiguous range of original Markdown source exposed by parsing as literal characters with no intervening Markdown escape, character reference, syntax - token, or node boundary. Each run retains its enclosing Markdown context for eligibility checks. - -**Eligible text** -: A literal-source run whose enclosing context is classified as textual content by GFM 0.29-gfm or explicitly exposed as ordinary text by a Memos - Markdown extension. - -**Opaque Markdown node** -: A GFM syntax node or Memos extension node whose source is not eligible for tag recognition. New extension node types are opaque by default. - -**Source spelling** -: The exact source substring consumed after the introducer, including ignored default-ignorable code points and ignored leading combining marks. Inline - rendering and source-preserving operations use this spelling. - -**Recognized source span** -: The exact contiguous substring of original Markdown formed by the introducer and its complete source spelling. - -**Direct tag value** -: The identifier value emitted from a tag occurrence before hierarchy expansion. It may differ from the source spelling because default-ignorable code - points outside a matched fully-qualified emoji sequence and ignored leading combining marks before the starter of each segment are consumed but omitted. - -**Implied ancestor tag** -: A slash-delimited prefix derived from a direct tag value. For example, `book` is an implied ancestor of the direct value `book/fiction`. - -**Tag segment** -: A non-empty component of a hierarchical tag identifier. Slashes separate segments and are not part of any segment. - -**Apostrophe joiner** -: U+0027 APOSTROPHE (`'`) or U+2019 RIGHT SINGLE QUOTATION MARK (`’`) emitted inside a tag segment only when, after emoji-first tokenization, the - immediately preceding source code point emits as an `XID_Continue` code point and the immediately following source code point emits as a non-combining - `XID_Continue` code point. - -**Display value** -: The direct or implied value presented as a derived tag label. It preserves emitted code points exactly but does not contain ignored default-ignorable code - points or ignored leading combining marks. - -**Comparison key** -: The value used to compare two display values for tag identity. It is the exact display value without case folding or Unicode - normalization. - -**Memo tag set** -: The union of direct tag values and their implied ancestor tags exposed for a memo. Each exactly equal value appears once; any difference in the value's - Unicode code-point sequence produces a separate membership. - -**Tag metadata rule** -: User configuration that selects tag values and supplies presentation or behavior metadata. Creating or changing a metadata rule does not create or rename - a tag. - -## Decision - -### Domain ownership - -Memo Markdown is the source of truth for tags. Stored payload tags, API fields, counts, navigation entries, completion candidates, and metadata matches are -derived projections of recognized source occurrences. - -There is no independently mutable tag resource in this domain model. In particular: - -- A tag cannot be renamed without editing every memo source occurrence that should change. -- Tag metadata may decorate or affect matching derived values, but it cannot change their source spelling or identity. -- Persisted payload tags are rebuildable indexes, not authoritative tag records. -- Import and export preserve memo Markdown; they do not substitute a separate canonical tag label for the spelling in source. - -### Lexical syntax - -Memos tag syntax is based on Unicode UAX #31 and Unicode Emoji data, with custom rules for Memos. It is not an unchanged implementation of UAX31-R8 or -the UAX #31 Emoji Profile. - -The normative grammar is: - -```text -TagCandidate := Introducer TagSourceSpelling -Introducer := U+0023 NUMBER SIGN ("#") outside a FullyQualifiedEmoji -TagSourceSpelling := TagSegmentSpelling ("/" TagSegmentSpelling)* -TagSegmentSpelling := IgnoredPrefix* SegmentStarter SegmentContinuation* -IgnoredPrefix := IgnoredDefaultIgnorable | IgnoredLeadingCombiningMark -SegmentContinuation := ValueUnit | ApostropheJoiner | IgnoredDefaultIgnorable - -SegmentStarter := EmittedXIDContinueCodePointExceptCombiningMark - | FullyQualifiedEmoji - | "-" - | "+" - | "&" - -ValueUnit := EmittedXIDContinueCodePoint - | FullyQualifiedEmoji - | "-" - | "+" - | "&" - -ApostropheJoiner := U+0027 APOSTROPHE ("'") - | U+2019 RIGHT SINGLE QUOTATION MARK ("’") - -IgnoredDefaultIgnorable := Default_Ignorable_Code_Point outside a FullyQualifiedEmoji -IgnoredLeadingCombiningMark := XID_Continue with General_Category Mn or Mc, - minus Default_Ignorable_Code_Point, before SegmentStarter -``` - -The grammar recognizes source spelling and emits a tag identifier as follows: - -- Each `SegmentStarter`, `ValueUnit`, and contextually valid `ApostropheJoiner` emits its exact source code-point sequence. A `FullyQualifiedEmoji` - therefore emits its complete matched sequence. -- Each consumed `/` emits one U+002F SOLIDUS into the identifier. -- `Introducer`, `IgnoredDefaultIgnorable`, and `IgnoredLeadingCombiningMark` emit nothing. - -At every source position, the token priority is the longest `FullyQualifiedEmoji`, then `IgnoredDefaultIgnorable`, then `IgnoredLeadingCombiningMark`, then -a contextually valid `ApostropheJoiner`, then an emitted code point or Memos extension unit. This preserves default-ignorable code points inside matched -fully-qualified emoji sequences while omitting them everywhere else. - -The emitted tag identifier is the direct tag value. - -Rules: - -1. `EmittedXIDContinueCodePoint` is a Unicode code point with the `XID_Continue` property in Unicode 17.0, minus - `Default_Ignorable_Code_Point` in Unicode 17.0. -2. `EmittedXIDContinueCodePointExceptCombiningMark` additionally excludes General Category `Mn` and `Mc` code points. -3. `_` is already included in `XID_Continue` and is not a separate extension. -4. `FullyQualifiedEmoji` is a code-point sequence whose `RGI_Emoji_Qualification` value is `Fully_Qualified`, represented by status `fully-qualified` in - the Emoji 17.0 `emoji-test.txt` data. It is matched atomically and before any shorter unit. The `Standalone_Component` value, represented by status - `component`, is not included. -5. The raw union of `Extended_Pictographic` and `Emoji_Component` is not used. That union admits bare `*`, isolated emoji components, invisible emoji tag - characters, and code points reserved for future emoji. -6. U+0023 ASCII NUMBER SIGN is the only tag introducer, but a `#` already matched inside the fully-qualified keycap sequence `#️⃣` is an emoji unit rather - than an introducer. U+FE5F SMALL NUMBER SIGN and U+FF03 FULLWIDTH NUMBER SIGN are ordinary text. -7. An ordinary `#` that does not begin a `FullyQualifiedEmoji` terminates the current identifier and may introduce another tag. The `#` in `#️⃣` is - preserved as part of that atomic emoji value unit under the same longest-match rule as every other admitted emoji. -8. Numeric-only and emoji-only identifiers are valid. -9. `/` is a hierarchy separator and is consumed only between two non-empty segments. It cannot begin or end an identifier, and two `/` separators cannot be - adjacent. -10. `-`, `+`, and `&` are Memos-specific segment-unit extensions. They are allowed at any position, may repeat, and may form an entire segment. -11. Default-ignorable code points inside a `FullyQualifiedEmoji` are preserved as part of that atomic value unit. Any other - `Default_Ignorable_Code_Point` is consumed as part of the source spelling but emits no code point into the direct tag value. Ignored code points do not - by themselves make a segment non-empty. -12. A non-default-ignorable `XID_Continue` code point whose General Category is `Mn` (Nonspacing Mark) or `Mc` (Spacing Combining Mark) is consumed but - omitted while it precedes the starter of its segment. The same non-default-ignorable code point is preserved as a value unit after that segment's - starter. -13. After applying the longest-`FullyQualifiedEmoji` token priority, U+0027 APOSTROPHE and U+2019 RIGHT SINGLE QUOTATION MARK are emitted as - `ApostropheJoiner` only when the immediately preceding source code point in the same segment was emitted as an `EmittedXIDContinueCodePoint` and the - immediately following source code point can emit as an - `EmittedXIDContinueCodePointExceptCombiningMark`. An apostrophe joiner therefore cannot start or end a segment, repeat without an intervening XID - code point, adjoin a fully-qualified emoji or Memos extension unit, or join across an ignored code point. -14. U+02BC MODIFIER LETTER APOSTROPHE (`ʼ`) is already an `XID_Continue` code point. It follows the ordinary XID rules rather than the contextual - apostrophe-joiner rule. -15. There is no tag-specific identifier length limit. The enclosing memo-size limit provides the resource bound. - -The grammar is intentionally broader than a programming-language identifier grammar. It does not require the first emitted unit to be `XID_Start`: -digits, `_`, `-`, `+`, `&`, and fully-qualified emoji can start a segment. Default-ignorable code points and non-default-ignorable `Mn` or `Mc` code points -may occur before each segment's starter but are omitted from the value. After the starter, those non-default-ignorable `Mn` and `Mc` code points are -preserved. ASCII and right-curly apostrophes are emitted only as contextual joiners between XID code points. A slash cannot begin a segment, and segments -made only from `-`, `+`, and `&` are explicitly valid. - -### Maximal-prefix scanning - -After finding an introducer, the lexer consumes the longest valid `TagSourceSpelling` within the current literal-source run: - -1. Before each segment's starter, consume default-ignorable code points and non-default-ignorable `XID_Continue` code points in General Category `Mn` or - `Mc` into the source spelling without emitting them. -2. Emit a segment starter, trying the longest matching `FullyQualifiedEmoji` before any shorter unit. -3. After the starter, preserve non-default-ignorable `XID_Continue` combining marks as ordinary value units; continue consuming default-ignorable code - points without emitting them. -4. Emit U+0027 or U+2019 as an apostrophe joiner only when, after emoji-first tokenization, the immediately preceding source code point emitted an XID - continuation unit and the immediately following source code point can emit a non-combining XID continuation unit. Otherwise stop before the - apostrophe. -5. Consume `/` only when the following source, after any ignored prefix, can emit a segment starter; then consume that segment. -6. Stop before a `/` that is leading, trailing, or followed only by an ignored prefix and then another `/`, a non-starter, or the end of the literal-source - run, leaving that slash and the remaining source unconsumed. -7. Otherwise stop before the first code point that cannot begin a valid continuation unit. -8. Keep the valid prefix already consumed; a later invalid character does not invalidate it. -9. Produce no candidate if the first `TagSegmentSpelling` cannot match. - -### Candidate enumeration - -Candidate discovery and identifier scanning use the same token priority: - -1. Match the longest `FullyQualifiedEmoji` at the current position before interpreting any constituent code point separately. Outside an active tag - candidate, advance past the complete sequence. -2. If no emoji sequence matches and the current code point is an introducer, scan a tag candidate. -3. After a successful candidate, continue at the first unconsumed source position. An ordinary terminating `#` is therefore reconsidered as a possible next - introducer. -4. After a failed introducer, advance by one code point so an adjacent `#` is still considered. -5. Otherwise, advance by one code point. - -This produces: - -```text -#️⃣ -> no tag; the source is one emoji -##️⃣ -> #️⃣ -#first#️⃣ -> first#️⃣ -#first#second -> first, second -##tag -> tag -``` - -Examples: - -| Source | Identifier | Remaining source | -| --- | --- | --- | -| `#foo,bar` | `foo` | `,bar` | -| `#foo.bar` | `foo` | `.bar` | -| `#foo:bar` | `foo` | `:bar` | -| `#x=y` | `x` | `=y` | -| `#price€` | `price` | `€` | -| `#C++` | `C++` | empty | -| `#R&D` | `R&D` | empty | -| `#tag's` | `tag's` | empty | -| `#сім'я` | `сім'я` | empty | -| `#O’Brien` | `O’Brien` | empty | -| `#café's` | `café's` | empty | -| `#users'` | `users` | `'` | -| `#foo'1️⃣` | `foo` | `'1️⃣` | -| `#'tag` | none | `'tag` | -| `#-foo` | `-foo` | empty | -| `#foo-` | `foo-` | empty | -| `#---` | `---` | empty | -| `#&&` | `&&` | empty | -| `#A‍B` | `AB` | empty | -| `#‍foo` | `foo` | empty | -| `#A‌B` | `AB` | empty | -| `#‌foo` | `foo` | empty | -| `#A️B` | `AB` | empty | -| `#́foo` | `foo` | empty | -| `#́` | none | U+0301 COMBINING ACUTE ACCENT | -| `#café` | `café` | empty | -| `#work/notes` | `work/notes` | empty | -| `#foo/́bar` | `foo/bar` | empty | -| `#book/` | `book` | `/` | -| `#/book` | none | `/book` | -| `#book//fiction` | `book` | `//fiction` | -| `#book/fiction/` | `book/fiction` | `/` | -| `#foo\+bar` | `foo` | `\+bar` | -| `#R&D` | `R` | `&D` | - -For the source below, the identifier is `foo`; the inline-code span begins immediately afterward: - -```markdown -#foo`test` -``` - -### Introducer boundary - -There is no general character boundary before the introducer. The following all contain the identifier `tag`: - -```text -#tag -hello#tag -中文#tag -``` - -This intentionally differs from the flowing-text boundary recommendation in UAX #31. Memos can distinguish URL fragments using its Markdown parser and -does not need to approximate URLs by forbidding a letter or number before `#`. - -An ordinary `#` is a hard boundary between identifiers: - -```text -#first#second -> first, second -##tag -> tag, introduced by the second # -## tag -> no tag; this is an ATX heading marker followed by text -``` - -### Markdown-context validation - -Markdown parsing first divides the original source into literal-source runs and records each run's enclosing context. Lexical scanning then finds candidates -within, but never across, those runs. A candidate becomes a tag occurrence only when its complete source span is in eligible text under GFM 0.29-gfm plus -the declared Memos extensions. - -The lexer reads literal characters from each literal-source run's original source range. It does not scan a decoded or rendered text-node value. A Markdown -escape, character reference, or other syntax node is a hard boundary that a candidate cannot cross. - -Tag occurrences are allowed in text contained by formatting and block structure, including emphasis, strong emphasis, strikethrough, headings, lists, -quotes, and table cells. - -Tag occurrences do not arise from source that is: - -- Escaped by Markdown, as in `\#tag`. -- Inside inline code, fenced code, or indented code. -- Anywhere inside a Markdown link, autolink, GFM literal URL, link reference, image, or image alt text. -- Inside raw HTML syntax or an opaque raw HTML block. -- Inside an opaque Memos extension node, including inline or block math. - -Consequently, a character reference that renders as `#` is not an introducer, and syntax that renders as an otherwise valid tag unit does not join the -literal source on either side: - -```text -#tag -> no tag -#tag -> no tag -#foo\+bar -> foo -#R&D -> R -#R&D -> R&D -``` - -The complete link-like node is excluded, not only its destination. Therefore neither the label nor destination produces a tag: - -```markdown -[release #notes](https://example.com/releases#notes) -``` - -Context is determined by the Markdown syntax tree or AST, not by a second URL regular expression. This has two important consequences: - -```text -https://example.com/path#tag -> no tag when parsed as a URL/autolink -/path#tag -> tag when the text is not parsed as a link -``` - -Text between inline HTML tags may still contain a tag when the Markdown parser exposes that text as an ordinary text node. The HTML markup itself and -opaque HTML blocks are never scanned. - -### Extracted value and source span - -- The extracted direct tag value contains the emitted identifier only, never the introducer. -- The recognized source span contains the introducer and the entire consumed source spelling. -- Highlighting and inline rendered tag markup use that full source span and preserve its source spelling; derived tag labels use the display value. -- Completion replaces the consumed source spelling after the introducer; it does not replace preceding prose. -- The original source spelling is preserved. Case folding and normalization must not occur before lexical boundaries are found. -- A default-ignorable code point outside a matched fully-qualified emoji sequence or an ignored leading combining mark before any segment's starter belongs - to the recognized source span but is omitted from the extracted value. - -For example, `##tag` extracts the value `tag`, while the recognized source span is the second `#tag`. - -For `#A‍B`, the recognized source span contains `#`, `A`, U+200D ZWJ, and `B`, while the extracted direct value is the two-code-point string `AB`. - -For `#́foo`, the recognized source span includes the leading U+0301 COMBINING ACUTE ACCENT, while the extracted direct value is `foo`. In `#café`, the -same U+0301 follows a segment starter and remains part of the extracted value. - -### Identifier equality - -The comparison key is the emitted display value. Two identifiers are equal only when their emitted Unicode code-point sequences are identical. Source -spellings that differ only by ignored default-ignorable code points or ignored leading combining marks therefore compare equal. - -Memos does not apply case folding, canonical normalization, compatibility normalization, width folding, locale-sensitive comparison, or accent folding when -deduplicating, counting, filtering, navigating, or performing exact metadata lookup. Consequently, each pair below contains two distinct tags: - -```text -#Work / #work -#café / #café -#A / #A -#straße / #STRASSE -#O'Brien / #O’Brien -#O’Brien / #OʼBrien -``` - -Multiple occurrences that emit exactly equal direct tag values in one memo produce one memo-tag membership. A tag metadata rule may deliberately match -several distinct tags, but that rule does not change their identity. - -Lexical emission applies only when recognizing Markdown source. An operand supplied to an exact filter or metadata lookup is already a tag value: it is -compared as supplied and is not re-lexed or stripped of code points. - -For example, `#AB`, `#A‍B`, and `#A‌B` all emit the comparison key `AB`; `#foo` and `#́foo` both emit `foo`. Each group contributes one tag membership if its -spellings occur in the same memo. Their Markdown source remains unchanged. - -### Hierarchy and memo membership - -The slash `/` is a structural hierarchy separator, not an opaque character in a tag value. A direct tag value contributes itself and every slash-delimited -ancestor prefix to the memo tag set: - -```text -Source occurrence: #book/fiction/history -Direct tag value: book/fiction/history -Memo tag set: book, book/fiction, book/fiction/history -``` - -Hierarchy expansion has the following domain semantics: - -- `Memo.tags` exposes both direct values and implied ancestors. -- Exact membership filtering for an ancestor matches a memo containing only a descendant occurrence. -- A tag count is the number of memo tag sets containing that value. One memo contributes at most one count to each direct or implied tag. -- Rendering and export preserve the one source occurrence; hierarchy expansion does not insert ancestor hashtags into Markdown. - -Every hierarchy segment is non-empty. A leading slash produces no candidate; a trailing or repeated slash terminates the identifier before that slash under -the maximal-prefix rule. - -### Pinned Unicode and emoji data - -Tag recognition is pinned to Unicode 17.0 and Emoji 17.0. Go, browser, and Node runtime tables are not implicitly normative. - -The pinned Unicode property assignments and Emoji data are normative, rather than the tables supplied by a particular runtime. Updating either data set -requires an explicit specification change because newly assigned `XID_Continue` characters and newly fully-qualified emoji sequences change what source -text means. Effects on existing data after such an update are outside scope. - -## Conformance examples - -The following examples are normative for the lexical and context decisions already made: - -| Source | Extracted identifiers | Reason | -| --- | --- | --- | -| `#tag` | `tag` | Basic identifier | -| `hello#tag` | `tag` | No general left boundary | -| `#标签` | `标签` | Multilingual XID characters | -| `#2026` | `2026` | Numeric-only identifiers are valid | -| `#C++` | `C++` | Explicit `+` extension | -| `#R&D` | `R&D` | Explicit `&` extension | -| `#tag's` | `tag's` | ASCII apostrophe joins two XID code points | -| `#сім'я` | `сім'я` | ASCII apostrophe preserves a Ukrainian word | -| `#O’Brien` | `O’Brien` | Right single quotation mark joins two XID code points | -| `#café's` | `café's` | An emitted combining mark may precede an apostrophe joiner | -| `#users'` | `users` | A trailing apostrophe is not a joiner | -| `#foo'1️⃣` | `foo` | Emoji-first tokenization prevents an apostrophe from adjoining the keycap sequence | -| `#'tag` | none | An apostrophe cannot start a segment | -| `'#tag'` | `tag` | Surrounding quotation punctuation remains outside the occurrence | -| `#rock’n’roll` | `rock’n’roll` | Multiple apostrophe joiners are valid when each independently satisfies the context rule | -| `#OʼBrien` | `OʼBrien` | U+02BC is an ordinary `XID_Continue` code point | -| `#` followed by 101 `a` code points | all 101 `a` code points | There is no tag-specific length limit | -| `#-foo` | `-foo` | Visible connector extensions may begin a segment | -| `#foo-` | `foo-` | Visible connector extensions may end a segment | -| `#---` | `---` | A segment may contain only visible connector extensions | -| `#&&` | `&&` | Visible connector extensions may repeat | -| `#A‍B` | `AB` | A non-emoji ZWJ is consumed but omitted from the value | -| `#‍foo` | `foo` | An ignored leading ZWJ does not prevent a later value unit from starting the segment | -| `#A‌B` | `AB` | A non-emoji ZWNJ is consumed but omitted from the value | -| `#‌foo` | `foo` | An ignored leading ZWNJ does not prevent a later value unit from starting the segment | -| `#‍‌` | none | Ignored default-ignorable code points alone do not form a segment | -| `#A️B` | `AB` | A default-ignorable variation selector outside a matched fully-qualified emoji is omitted even after a starter | -| `#́foo` | `foo` | A non-default-ignorable combining mark before the segment starter is consumed but omitted | -| `#́` | none | Leading combining marks alone do not form a segment | -| `#café` | `café` | A non-default-ignorable combining mark after the segment starter is preserved | -| `#work/notes` | `work/notes` | Direct hierarchical identifier; memo membership also includes `work` | -| `#foo/́bar` | `foo/bar` | Ignored leading combining-mark handling restarts for each segment | -| `#book/` | `book` | A trailing slash is not consumed | -| `#/book` | none | A hierarchy cannot begin with an empty segment | -| `#book//fiction` | `book` | A repeated slash terminates before the first slash | -| `#book/fiction/` | `book/fiction` | The valid hierarchical prefix is retained | -| `#l·l` | `l·l` | Middle dot is `XID_Continue` | -| `#foo‿bar` | `foo‿bar` | Connector punctuation is `XID_Continue` | -| `#*️⃣` | `*️⃣` | Fully-qualified keycap sequence | -| `#‼️` | `‼️` | Fully-qualified emoji sequence | -| `#♥` | none | Bare text-presentation symbol is neither XID nor fully-qualified emoji | -| `#♥️` | `♥️` | Fully-qualified emoji sequence | -| `#🏻` | none | A standalone emoji component is excluded from `FullyQualifiedEmoji` | -| `#foo,` | `foo` | Comma terminates the identifier | -| `#price€` | `price` | Currency symbol terminates the identifier | -| `#€budget` | none | The first body character is not a unit | -| `#v²` | `v` | `Other_Number` is not generally `XID_Continue` | -| `#first#second` | `first`, `second` | Ordinary `#` terminates and starts candidates | -| `#️⃣` | none | The complete source is a fully-qualified emoji, not an introducer plus identifier | -| `##️⃣` | `#️⃣` | The first `#` introduces the atomic keycap emoji value | -| `#first#️⃣` | `first#️⃣` | The keycap emoji is an ordinary fully-qualified continuation unit | -| `#tag` | none | Fullwidth number sign is not an introducer | -| `﹟tag` | none | Small number sign is not an introducer | -| `#tag` | none | A character reference is not a literal introducer | -| `#tag` | none | Decoded text is not rescanned for candidates | -| `\#tag` | none | Escaped introducer | -| `#foo\+bar` | `foo` | A Markdown escape terminates the literal-source range | -| `#R&D` | `R` | A character reference terminates the literal-source range | -| `#R&D` | `R&D` | A literal ampersand is a value unit | -| `` `#tag` `` | none | Inline-code context | -| `**#urgent**` | `urgent` | Formatted normal text remains eligible | -| `https://example.com/#tag` | none | GFM URL/autolink context | -| `/path#tag` | `tag` | Plain text path is not guessed to be a URL | -| `[hello#tag](https://example.com)` | none | The complete link is excluded | -| `$#tag$` | none | A Memos math node is opaque | - -## Consequences - -### Positive - -- Tag meaning is specified independently of any single parser or issue. -- Common multilingual tags, numeric tags, hierarchy characters, and emoji remain expressive. -- Hierarchical ancestors have one consistent meaning across API membership, exact filters, navigation, and counts. -- `hello#tag` works consistently while actual Markdown links and URLs are excluded structurally. -- Common word-internal apostrophes support multilingual words and names while surrounding quotation punctuation remains outside tag values. -- Unsupported Markdown punctuation, delimiters, currency symbols, and operators stop an identifier predictably; apostrophes are the explicitly - constrained exception. -- Exact equality preserves every emitted code-point distinction and avoids locale-dependent identity rules. -- Ignoring default-ignorable code points outside emoji avoids invisible tag distinctions while preserving matched fully-qualified emoji sequences. -- Ignoring leading non-default-ignorable `Mn` and `Mc` code points only before each segment's visible starter avoids invisible-leading segments without - breaking decomposed writing inside words. -- Keeping `-`, `+`, and `&` position-independent avoids a second connector-validation layer. -- The memo-size limit provides the only length bound, avoiding tag-specific counting and overflow rules. -- All consumers can implement the same language contract. -- Unicode repertoire changes require an explicit specification update rather than depending on runtime tables. - -### Negative - -- Fully-qualified emoji matching requires sequence-aware data, not a simple code-point character class. -- Pinning Unicode data requires maintenance when Unicode and Emoji data are updated. -- Visually indistinguishable or canonically equivalent source spellings whose emitted values differ remain separate tags unless the user edits their memo - sources to make those values identical. -- Visually similar apostrophe spellings such as U+0027, U+2019, and U+02BC remain distinct under exact tag identity. -- English possessive-looking source such as `#tag's` denotes the complete tag value `tag's`, not `tag` followed by prose. -- Source spellings that differ only by ignored default-ignorable code points intentionally collapse to the same emitted tag value. -- Source spellings that differ only by ignored leading combining marks intentionally collapse to the same emitted tag value. - -## Alternatives considered - -### Keep `L/M/N/S` and patch URL detection - -Rejected. `\p{S}` admits currency, mathematical operators, modifier symbols, and Markdown backticks merely as a side effect of supporting emoji. URL detection -would still differ across consumers and would duplicate the Markdown parser. - -### Require a whitespace or punctuation boundary before `#` - -Rejected. It makes the intentionally supported `hello#tag` form invalid and uses a lexical approximation to solve a structural URL problem. - -### Adopt UAX31-R8 unchanged - -Rejected. UAX31-R8 recommends a flowing-text left boundary that does not match the product decision, and its raw emoji component repertoire is broader than -the desired complete-sequence behavior. Memos instead defines an explicit UAX #31 profile. - -### Use `Extended_Pictographic | Emoji_Component` as code-point characters - -Rejected. It admits bare `*`, isolated modifiers and regional indicators, invisible emoji tag characters, and unassigned code points reserved for future -emoji. Matching only fully-qualified entries expresses the intended user-visible unit. - -### Keep the 100-code-point limit - -Rejected. The value is arbitrary, counts code points rather than user-perceived characters, and currently has three different overflow behaviors. Memo-size -limits already bound work. - -### Reject an entire candidate when a later invalid character appears - -Rejected. Hashtags in flowing text conventionally end at punctuation. Maximal-prefix behavior makes `#foo,` and `#foo` followed by inline code predictable. - -### Allow empty hierarchy segments - -Rejected. Empty segments have no useful navigation or membership meaning and would require special handling in hierarchy expansion. The grammar permits `/` -only between two non-empty segments. - -### Reject a whole hierarchy after an invalid slash - -Rejected. Keeping the longest valid prefix makes a trailing or repeated slash behave like other terminating syntax and avoids erasing an otherwise valid -tag. The slash and remaining source stay ordinary Markdown. - -### Restrict visible connectors by position - -Rejected. Requiring `-`, `+`, or `&` to be medial, non-repeating, or accompanied by a letter would add validation rules without resolving a structural -ambiguity. They remain ordinary segment units; only `/` has structural meaning. - -### Treat apostrophes as unrestricted value units - -Rejected. Allowing apostrophes as starters, trailing units, or repeatable ordinary units would absorb surrounding quotation punctuation into values such -as `tag'` and make quoted source such as `'#tag'` ambiguous. The contextual joiner rule supports words and names while leaving punctuation at tag -boundaries unconsumed. - -### Recognize compatibility number signs - -Rejected. U+FE5F SMALL NUMBER SIGN and U+FF03 FULLWIDTH NUMBER SIGN are visually similar to `#` but are not Markdown syntax. Recognizing only ASCII U+0023 -keeps the introducer rule literal and unambiguous. - -### Special-case the number-sign keycap - -Rejected. `#️⃣` follows the same longest-sequence matching rule as every other admitted fully-qualified emoji. Excluding it or splitting its code points -would add an exception to the emoji profile. - -### Scan decoded Markdown text - -Rejected. Decoding character references and escapes before tag scanning would allow non-literal introducers and require source reconstruction for exact -spans. Tag lexing uses original source ranges; Markdown syntax nodes act as boundaries. - -### Leave the Markdown dialect parser-defined - -Rejected. Parser-dependent context would allow identical source to produce different tags. GFM 0.29-gfm is the structural baseline; additional Memos -extensions are opaque unless explicitly declared transparent to tag recognition. - -## References - -- [Unicode 17.0 Standard Annex #31: Unicode Identifiers and Syntax](https://www.unicode.org/reports/tr31/tr31-43.html) -- [UAX #31 Hashtag Identifiers](https://www.unicode.org/reports/tr31/tr31-43.html#Hashtag_Identifiers) -- [UAX #31 Emoji Profile](https://www.unicode.org/reports/tr31/tr31-43.html#Emoji_Profile) -- [Unicode Emoji 17.0 (UTS #51)](https://www.unicode.org/reports/tr51/tr51-29.html) -- [Unicode Emoji 17.0 test data](https://www.unicode.org/Public/17.0.0/emoji/emoji-test.txt) -- [GitHub Flavored Markdown specification](https://github.github.com/gfm/) -- [Memos tag documentation](https://usememos.com/docs/usage/tags) diff --git a/docs/adr/0002-username-format-and-references.md b/docs/adr/0002-username-format-and-references.md deleted file mode 100644 index be683ae2d..000000000 --- a/docs/adr/0002-username-format-and-references.md +++ /dev/null @@ -1,260 +0,0 @@ -# ADR 0002: Username Format and References - -Status: Accepted - -Date: 2026-08-02 - -Domain glossary: [Memos domain glossary](../glossary.md) - -## Context - -A username is a user identifier used by account creation and updates, authentication, public user resource names, API lookup, SSO provisioning, and -references to users. Those uses must share one format rather than defining separate username-like tokens. - -Memos already restricts newly written usernames to ASCII letters, digits, and hyphens, with a 36-character limit and alphanumeric ends. Existing -installations may also contain older usernames such as email addresses or values with underscores. Memos continues to read and resolve those legacy -values, but they are not valid new usernames. - -Markdown mentions currently drift from the writable username format: the backend accepts up to 63 Unicode letters, numbers, and hyphens; the frontend -accepts up to 63 ASCII characters; both allow invalid username shapes; and extraction lowercases the result. Editor decoration also scans raw source in -contexts where Markdown must be opaque. - -This ADR defines: - -- The format and case-preservation semantics for writable usernames. -- How username references resolve. -- Markdown mention as one source form of username reference. - -Changing or migrating legacy username data is outside this ADR's scope. -Username rename and reuse policy, and the durable binding of an already resolved mention across those lifecycle events, are also outside scope. -Notification timing, deduplication, access policy, self-mention behavior, and other downstream effects are unchanged and outside scope. - -## Decision drivers - -- Give every username write path one small, stable validation rule. -- Make a reference use the username format instead of inventing another identifier grammar. -- Preserve the spelling and case selected by the user. -- Keep username write validation, reference rendering, extraction, editor recognition, and notification resolution aligned. -- Use parsed Markdown context rather than special-case regular expressions for code, links, email addresses, and other opaque syntax. -- Keep recognition deterministic and linear in memo size. - -## Terminology - -**User** -: A durable Memos account identified internally by an immutable user ID. - -**Username** -: A user-selected, case-sensitive public identifier used for account addressing. A username is distinct from a display name and from the user's internal - ID. - -**Writable username** -: A username accepted when an account is created, renamed, or provisioned after this decision. - -**Legacy username** -: A stored username that does not satisfy the writable format but remains readable and addressable for compatibility. - -**Username reference** -: Source text that names a username so Memos can attempt to resolve it to a user ID. The source spelling is not itself a durable relationship. - -**Mention candidate** -: A Markdown source span with the form `@` followed by a complete writable username and satisfying the mention boundary and context rules. - -**Resolved mention** -: A mention candidate whose username resolves to a user under the consuming operation's existing account-status and visibility policy. - -## Decision - -### Writable username format - -The normative grammar is: - -```text -Username := Alphanumeric - | Alphanumeric UsernameCharacter{0,34} Alphanumeric -UsernameCharacter := Alphanumeric | "-" -Alphanumeric := ASCII letter | ASCII digit -ASCII letter := "A".."Z" | "a".."z" -ASCII digit := "0".."9" -``` - -- A username is 1 through 36 ASCII characters. -- ASCII letters, digits, and `-` are the only admitted characters. -- The first and last characters are letters or digits. -- Consecutive interior hyphens are valid; there is no additional rule limiting their number. -- A digits-only value is valid. -- Case is preserved. - -Requiring alphanumeric ends also excludes values made only of hyphens and prevents a trailing hyphen from becoming an ambiguous-looking part of a -Markdown mention. - -The minimum is one because Username needs no product-level namespace-scarcity policy beyond being non-empty. The maximum is 36 so every UUID value has a -canonical textual representation that fits while public URLs and reference tokens remain bounded. - -The ASCII-only restriction is intentional: Username is a machine-facing public identifier used in URLs, authentication, and references. Internationalized -human-readable naming belongs to Display Name rather than expanding Username normalization and comparison rules. - -Examples: - -| Value | Writable | Reason | -| --- | --- | --- | -| `alice` | yes | Letters only | -| `Alice-2` | yes | Case is preserved; interior hyphen is valid | -| `1alice` | yes | A digit may begin a username that also contains a letter | -| `a--b` | yes | Consecutive interior hyphens are valid | -| `a---b` | yes | Interior hyphens have no repetition limit | -| `123` | yes | Numeric spelling has no special meaning | -| `123-456` | yes | Digits and interior hyphens satisfy the grammar | -| `00000000-0000-0000-0000-000000000000` | yes | A canonical UUID occupies 36 characters | -| `-alice` | no | A hyphen cannot begin a username | -| `alice-` | no | A hyphen cannot end a username | -| `alice_smith` | no | `_` is outside the format | -| `alice@example.com` | no | Email syntax is outside the format | -| `álîçé` | no | The writable format is ASCII | -| `张三` | no | Internationalized naming belongs to Display Name | - -All account creation, username update, and automatic provisioning paths use this rule. A generic resource-name rule is not the username contract and -must not be reused as a substitute merely because its current regular expression is similar. - -### Username allocation - -The writable format has no reserved spellings. Values such as `admin`, `root`, `system`, `api`, `memos`, and `support` are ordinary usernames and are -available subject to the same exact-case uniqueness rule as every other value. A username does not grant authority or establish trust; those properties -come from the resolved user ID, role, and explicit product presentation. Manual registration, rename, and automatic provisioning apply the same policy. - -If Memos later needs a system-owned username, the corresponding account claims it through ordinary uniqueness rather than expanding a lexical reserved -list. - -SSO provisioning adopts the external identifier exactly when it is a valid, available writable username. If it is invalid or already belongs to another -user, provisioning assigns a generated canonical UUID instead. A matching username never links or takes over an account; only the persisted identity -binding formed by the identity-provider key and external subject identifies a returning SSO user. - -### Username identity and resolution - -Username spelling is case-preserving and username equality is exact ASCII byte equality. Memos does not lowercase, case-fold, or otherwise normalize a -username before validation, uniqueness checks, authentication, resource lookup, or reference resolution. In particular, surrounding whitespace is not -trimmed into a different username; a value such as ` alice ` is invalid. `Alice` and `alice` are distinct usernames and may identify different users. - -All storage backends and application lookup paths must preserve this equality rule. A database collation that treats `Alice` and `alice` as equal, or a -lookup that silently resolves one spelling to the other, does not conform to this decision. - -The `username` column therefore uses an explicit case-sensitive binary collation on every supported storage backend: `utf8mb4_bin` on MySQL, `C` on -PostgreSQL, and `BINARY` on SQLite. For the writable ASCII format, this makes exact-byte uniqueness and lookup a schema property rather than an assumption -about each database's default collation. - -A username-bearing field or resource-name segment always resolves as a username, including when its spelling contains only digits. Character shape must -not select the internal user-ID namespace; an operation that addresses a user by internal ID must expose that choice separately. - -The stable identity of a user is the internal user ID. A username reference is resolved as late as the consuming operation permits: - -1. Preserve the username exactly as written. -2. Look up that exact username in the operation's visibility and account-status scope. -3. If it resolves, use the resulting user ID for durable effects. -4. If it does not resolve, retain ordinary source text and produce no user-targeted effect. - -Product intent permits username rename and later reuse while requiring an already resolved mention not to change users. A source spelling alone cannot -satisfy all three properties: preserving the original target requires a durable binding to the user ID outside the `@username` text. This ADR does not -define that binding, historical re-resolution behavior, or the username lifecycle needed to support it. - -Legacy usernames remain valid exact lookup keys for existing API resource names and authentication flows. They are not accepted by write validation and -are not added to new reference syntaxes. Changing legacy username values remains outside this decision, but every lookup preserves their stored spelling. - -### Markdown mention syntax - -A Markdown mention is the first defined username-reference syntax: - -```text -MentionCandidate := "@" Username -``` - -The `Username` production is the complete writable username format above. Mention parsing has no independent character set or length limit. - -Candidate recognition follows these rules: - -1. The `@` is U+0040 COMMERCIAL AT in an eligible literal-source run. -2. At the start of the document, the left boundary is satisfied. Otherwise the immediately preceding source character must not be a - `UsernameCharacter`. The username format therefore defines the boundary: a preceding letter, digit, or hyphen blocks recognition. -3. The lexer consumes the complete consecutive run of ASCII letters, digits, and hyphens after `@`, then validates that whole run as a writable username. - It does not shorten an invalid run into a valid prefix. -4. GFM email-address recognition takes precedence. No substring of a recognized email address is a mention. - -Characters outside `UsernameCharacter`, including `_`, `@`, and non-ASCII characters, are ordinary boundaries and receive no special handling. - -The rules produce: - -| Source | Mention candidate | -| --- | --- | -| `@alice` | `alice` | -| `@Alice-2` | `Alice-2` | -| `hi, @alice.` | `alice` | -| `中文@alice` | `alice` | -| `hello@alice` | none | -| `foo-@alice` | none | -| `foo_@alice` | `alice` | -| `@123` | `123` | -| `@-alice` | none | -| `@alice-` | none | -| `@alice_smith` | `alice` | -| `@alice@example` | `alice` | -| `@alice@bob` | `alice` | -| `@alice@example.com` | none | -| `@` followed by 37 valid username characters | none | - -### Markdown context - -Mention recognition operates on original Markdown source after GFM and Memos block and inline structure is resolved. It uses the same literal-source-run -and opaque-node model as other Memos inline extensions: - -- Paragraphs, headings, block quotes, list items, table text, emphasis, strong emphasis, and strikethrough expose ordinary text. -- Code spans, code blocks, resolved links and images, autolinks, recognized GFM email addresses, raw HTML, and math are opaque. -- Escapes, character references, line endings, and syntax delimiters end a literal-source run. An escaped or character-reference-produced `@` is not an - introducer. -- Unknown future Markdown extension nodes are opaque unless their definition explicitly exposes ordinary text. - -Parsing, read rendering, metadata extraction, editor decoration, and any future mention completion must use these same lexical and context rules. - -This decision changes only which exact username spellings are extracted as mention candidates. Existing resolution, rendering, notification, and access -policies consume those candidates without otherwise changing behavior. Autocomplete, suggestion ranking, and a dedicated structured mention payload are -also outside this decision. - -## Consequences - -- Username validation becomes a domain primitive shared by every write path. -- Username uniqueness, authentication, resource lookup, and reference resolution use the same case-sensitive equality rule on every storage backend. -- Mention syntax automatically follows future username-format decisions only when this ADR is superseded and both uses are updated together. -- Existing valid mixed-case usernames can be mentioned without lowercasing. -- Values accepted only by the old 63-character or Unicode mention lexers stop being mention candidates because they were never writable usernames. -- Legacy email-like or underscore-containing usernames remain usable through compatibility lookup but cannot be authored as new Markdown mentions. -- Markdown-aware recognition removes false mentions from code, links, email addresses, HTML, math, escapes, and character references. -- Username rename, reuse, and historical mention stability remain an explicit follow-up because source text alone cannot represent the intended stable - user binding. - -## Alternatives considered - -### Define mention syntax first - -Rejected because it creates a second username-like grammar and allows account validation and references to drift again. - -### Allow a broader mention token than writable usernames - -Rejected because such candidates cannot reliably identify newly writable accounts and complicate boundaries, validation, and UI messaging. - -### Store stable user IDs with a mention - -Deferred. Stable IDs make renames and username reuse robust but require either a new source representation or persisted occurrence bindings, plus editing -UX, import/export behavior, and compatibility policy. Exact username references are sufficient for candidate recognition in the current scope, but do not -solve historical binding. - -### Unicode usernames - -Deferred. A safe Unicode identifier design needs a pinned Unicode version, normalization, script and confusable policy, and consistent database collation. -ASCII keeps this decision small and matches the existing writable format. - -## References - -- [GitHub Flavored Markdown 0.29-gfm](https://github.github.com/gfm/) -- [GitHub: Mentioning people and teams](https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#mentioning-people-and-teams) -- [Slack message formatting: mentioning users](https://docs.slack.dev/messaging/formatting-message-text/#mentioning-users) -- [Matrix client-server API: user and room mentions](https://spec.matrix.org/latest/client-server-api/#user-and-room-mentions) -- [Activity Streams 2.0 Vocabulary: Mention](https://www.w3.org/TR/activitystreams-vocabulary/#dfn-mention) -- [Unicode Standard Annex #31: Unicode Identifiers and Syntax](https://www.unicode.org/reports/tr31/) -- [Unicode Technical Standard #39: Unicode Security Mechanisms](https://www.unicode.org/reports/tr39/) diff --git a/docs/adr/0003-space-uid-allocation-and-format.md b/docs/adr/0003-space-uid-allocation-and-format.md deleted file mode 100644 index c3fe69c57..000000000 --- a/docs/adr/0003-space-uid-allocation-and-format.md +++ /dev/null @@ -1,65 +0,0 @@ -# ADR 0003: Space UID Allocation and Format - -Status: Accepted - -Date: 2026-08-27 - -Domain glossary: [Memos domain glossary](../glossary.md) - -## Context - -A Space title is mutable and non-unique. Using it alone in switchers, settings, badges, and destructive confirmations makes distinct Spaces with the same -title difficult to tell apart. The immutable public identity is instead the instance-wide UID embedded in the resource name `spaces/{space UID}`. - -Previously, first-party clients omitted the UID and the server generated a short UUID. That prevented clients from choosing a stable identifier before -creation and made the default differ from other newer UUID-backed identities. Existing clients may still omit the request field, and existing short UIDs -must remain valid without a data migration. - -## Decision drivers - -- Give clients a stable identifier before they issue a create request. -- Keep old clients and existing Space resource names compatible. -- Reuse the established public-resource UID grammar instead of introducing a Space-only slug format. -- Keep the complete immutable UID discoverable in Settings without adding identity metadata to every compact Space label. - -## Decision - -The first-party client generates a canonical lowercase UUID v4 for every new Space and sends it in `CreateSpaceRequest.space_id`. It may expose that value -before creation and let the user replace it with a custom UID. A retry of the same create interaction reuses the same generated value. - -The API field remains optional for compatibility. When it is empty, the server generates a canonical lowercase UUID v4. A supplied value uses the shared -public-resource grammar: - -```text -SpaceUID := Alphanumeric - | Alphanumeric UIDCharacter{0,34} Alphanumeric -UIDCharacter := Alphanumeric | "-" -Alphanumeric := ASCII letter | ASCII digit -``` - -The UID is therefore 1 through 36 characters. Consecutive interior hyphens, uppercase letters, and digits-only values are valid. The one-character -minimum matches the other public-resource UIDs; increasing it would not materially prevent collisions or name claiming. The spelling is preserved. - -The UI uses the title as the primary label. Settings surfaces and their management subflows always show the complete UID with an explicit `Space UID` -label. Other surfaces show the UID only when two known Spaces have exactly matching, case-sensitive titles, or when the title is unavailable and the UID -is the only usable identity. When compact identity metadata is needed, canonical UUIDs use an eight-character prefix; short custom UIDs are shown in -full, while long custom UIDs show both ends so late differences remain visible. Accessible labels and tooltips retain the complete UID in those cases. - -Existing Space UIDs remain readable and are not rewritten. - -## Consequences - -- New first-party creates use UUID v4 by default, while custom identifiers remain concise and human-readable when desired. -- Old clients continue to work through the server fallback. -- Duplicate titles remain allowed and require no rename or uniqueness migration. -- Compact Space labels stay title-only until a matching title or missing-title fallback makes the UID necessary. -- UID collisions are rejected by the existing instance-wide uniqueness constraint. -- Case-preserving UID input is accepted. Aligning exact-case uniqueness and lookup across database collations is a separate schema decision. - -## Alternatives considered - -- **Require unique titles.** Rejected because titles are display labels and users may legitimately reuse them. -- **Always show UIDs across the UI.** Rejected because it makes a technical identifier compete with the human-readable title even when no ambiguity exists. -- **Only show UIDs in Settings.** Rejected because matching titles would remain ambiguous in switchers, badges, and scoped search. -- **Keep server-only allocation.** Rejected because the first-party client cannot retain one identity across retries or offer customization before creation. -- **Require a longer custom UID.** Rejected because length does not meaningfully solve collision or namespace-claiming concerns. diff --git a/docs/adr/README.md b/docs/adr/README.md deleted file mode 100644 index 1cab5e9ae..000000000 --- a/docs/adr/README.md +++ /dev/null @@ -1,62 +0,0 @@ -# Architecture Decision Records - -Architecture Decision Records (ADRs) document significant technical and product decisions whose rationale should remain available after the related -implementation work is complete. - -## Index - -| ADR | Title | Status | Date | -| --- | --- | --- | --- | -| [0001](0001-tag-syntax-and-recognition.md) | Tag Syntax and Recognition | Accepted | 2026-08-01 | -| [0002](0002-username-format-and-references.md) | Username Format and References | Accepted | 2026-08-02 | -| [0003](0003-space-uid-allocation-and-format.md) | Space UID Allocation and Format | Accepted | 2026-08-27 | - -## Conventions - -- Name files `NNNN-short-kebab-case-title.md`, using a four-digit number that is never reused. -- Assign the next number when an ADR is opened, even if an earlier ADR is later rejected or superseded. -- Use the title `# ADR NNNN: Title` and record dates as `YYYY-MM-DD`. -- Keep accepted ADRs as historical records. Replace a decision with a new ADR rather than rewriting the original rationale. -- When one ADR replaces another, add `Supersedes: ADR NNNN` to the new ADR and `Superseded by: ADR NNNN` to the old ADR. - -## Statuses - -- **Proposed**: Under discussion; unresolved questions may remain. -- **Accepted**: Approved as the decision to implement and maintain. -- **Rejected**: Considered but not selected. -- **Superseded**: Replaced by a later ADR. - -## Suggested structure - -Each ADR should contain enough information to understand the decision without reconstructing the original discussion: - -```markdown -# ADR NNNN: Title - -Status: Proposed - -Date: YYYY-MM-DD - -## Context - -## Decision drivers - -## Decision - -## Consequences - -## Alternatives considered - -## Open questions before acceptance - -## References -``` - -Optional sections may be omitted when they do not add useful context. - -## Process - -1. Create a `Proposed` ADR with the next available number and add it to the index. -2. Discuss the proposal with maintainers, updating the ADR as the decision develops. -3. Change the status to `Accepted` or `Rejected` when the outcome is clear. -4. If an accepted decision changes materially, create a new ADR and cross-link the superseding records. diff --git a/docs/design/README.md b/docs/design/README.md new file mode 100644 index 000000000..3d2bb2014 --- /dev/null +++ b/docs/design/README.md @@ -0,0 +1,16 @@ +# Design Documents + +Design documents may evolve while a feature is being designed and implemented. Once implementation is complete, set `Status: Implemented` and freeze the document. + +Do not update implemented designs to track later code changes. They preserve the original design and rationale; source code and tests describe current +behavior. New and unimplemented designs may still be added or revised. + +The following designs have been implemented and are frozen: + +- [API abuse controls](api-abuse-controls.md) +- [Configuration provisioning](configuration-provisioning.md) +- [Memos Export Format](memos-export-format.md), including the schemas under `memo-export/1.0/` +- [Multi-Spaces](multi-spaces.md) +- [Unique email](unique-email.md) + +The configuration provisioning design was moved here from `docs/configuration-provisioning.md` and frozen after implementation. diff --git a/docs/design/api-abuse-controls.md b/docs/design/api-abuse-controls.md index 10c5708be..a2c3b2da7 100644 --- a/docs/design/api-abuse-controls.md +++ b/docs/design/api-abuse-controls.md @@ -1,6 +1,6 @@ # API Abuse Controls Design -Status: Accepted +Status: Implemented Date: 2026-09-12 diff --git a/docs/configuration-provisioning.md b/docs/design/configuration-provisioning.md similarity index 100% rename from docs/configuration-provisioning.md rename to docs/design/configuration-provisioning.md diff --git a/docs/design/memo-export/1.0/manifest.schema.json b/docs/design/memo-export/1.0/manifest.schema.json index ee32333c2..03d920220 100644 --- a/docs/design/memo-export/1.0/manifest.schema.json +++ b/docs/design/memo-export/1.0/manifest.schema.json @@ -68,7 +68,7 @@ "username": { "type": "string", "minLength": 1, - "description": "Username as defined by ADR 0002." + "description": "Username identifying the user in the export." } } } diff --git a/docs/design/memo-export/1.0/memo.schema.json b/docs/design/memo-export/1.0/memo.schema.json index 47fd626ee..679e895b0 100644 --- a/docs/design/memo-export/1.0/memo.schema.json +++ b/docs/design/memo-export/1.0/memo.schema.json @@ -89,7 +89,7 @@ "username": { "type": "string", "minLength": 1, - "description": "Username as defined by ADR 0002." + "description": "Username identifying the user in the export." }, "timestamp": { "type": "string", diff --git a/docs/design/memos-export-format.md b/docs/design/memos-export-format.md index 873ba0f52..68a799d0b 100644 --- a/docs/design/memos-export-format.md +++ b/docs/design/memos-export-format.md @@ -1,12 +1,12 @@ # Memos Export Format -Status: Proposed +Status: Implemented Date: 2026-09-16 Existing domain language: [Memos context](../../CONTEXT.md) -Related: [Multi-Spaces Design](multi-spaces.md), [ADR 0002: Username Format and References](../adr/0002-username-format-and-references.md), [ADR 0003: Space UID Allocation and Format](../adr/0003-space-uid-allocation-and-format.md) +Related: [Multi-Spaces Design](multi-spaces.md) Normative schemas: [`memo-export/1.0/manifest.schema.json`](memo-export/1.0/manifest.schema.json), [`memo-export/1.0/memo.schema.json`](memo-export/1.0/memo.schema.json) @@ -81,8 +81,8 @@ Container rules: ### Identifiers -- A memo is identified by its Memo UID, the `{memo}` segment of `memos/{memo}`, matching `^[a-zA-Z0-9]([a-zA-Z0-9-]{0,34}[a-zA-Z0-9])?$`. An attachment is identified by the `{attachment}` segment of `attachments/{attachment}`, which uses the same grammar. A Space is identified by its Space UID under ADR 0003 and carries its Space title for fallback matching. Database identifiers never appear. -- A user is identified by username under ADR 0002. +- A memo is identified by its Memo UID, the `{memo}` segment of `memos/{memo}`, matching `^[a-zA-Z0-9]([a-zA-Z0-9-]{0,34}[a-zA-Z0-9])?$`. An attachment is identified by the `{attachment}` segment of `attachments/{attachment}`, which uses the same grammar. A Space is identified by its Space UID and carries its Space title for fallback matching. Database identifiers never appear. +- A user is identified by username. - References inside the archive are bare UIDs, never resource names, so the resource name patterns of the API can change without a format change. ### Records diff --git a/docs/design/multi-spaces.md b/docs/design/multi-spaces.md index 3752d2124..1a48e4383 100644 --- a/docs/design/multi-spaces.md +++ b/docs/design/multi-spaces.md @@ -1,6 +1,6 @@ # Multi-Spaces Design -Status: Accepted +Status: Implemented Date: 2026-08-23 diff --git a/docs/design/unique-email.md b/docs/design/unique-email.md index 615a9a19d..55be32069 100644 --- a/docs/design/unique-email.md +++ b/docs/design/unique-email.md @@ -1,12 +1,12 @@ # Unique Email Design -Status: Accepted +Status: Implemented Date: 2026-09-12 Existing domain language: [Memos context](../../CONTEXT.md) -Related: [Multi-Spaces Design](multi-spaces.md), [ADR 0002: Username Format and References](../adr/0002-username-format-and-references.md) +Related: [Multi-Spaces Design](multi-spaces.md) ## Summary diff --git a/docs/glossary.md b/docs/glossary.md deleted file mode 100644 index 6de8b29c0..000000000 --- a/docs/glossary.md +++ /dev/null @@ -1,145 +0,0 @@ -# Memos Domain Glossary - -This glossary defines product and domain language shared across Memos design documents. Protocol, Unicode, and parser-specific terms remain in the ADRs -that use them. - -## Users and usernames - -### User - -A durable Memos account. Its stable identity is an internal user ID, not its mutable username or display name. - -### Username - -A user-selected, case-sensitive public account identifier used for authentication, user resource names, lookup, and references. Username spelling is -case-preserving, is not implicitly trimmed or normalized, and equality is exact ASCII byte equality, so `Alice` and `alice` are distinct usernames. A -username is distinct from both a display name and the user's internal ID. Its spelling does not grant a role or establish that the user is trusted. - -### Writable username - -A username accepted for account creation, rename, or automatic provisioning. It is 1 through 36 ASCII letters, digits, or hyphens; begins and ends with an -ASCII letter or digit; and may contain repeated interior hyphens without an additional restriction. Numeric spelling has no special meaning, so a value -containing only digits is also a writable username. Internationalized human-readable naming belongs to the user's display name. - -### Legacy username - -A stored username that does not satisfy the writable format but remains readable and addressable for compatibility. Legacy usernames are not accepted by -new write validation or added to new reference syntaxes. - -### Username reference - -Source text that names an exact, case-sensitive username so Memos can attempt to resolve it to a user ID. The source spelling alone is not a durable user -binding. Stability across username rename and reuse requires additional persisted identity that the current username-reference specification does not -define. - -### Mention candidate - -An eligible memo Markdown source span formed by an ASCII `@` followed by one complete writable username. Recognition also applies the boundary and opaque -Markdown-context rules in the active username-reference specification. The same ASCII letters, digits, and hyphen admitted inside a username define the -mention boundaries: they block a mention immediately to their right and are consumed as part of the complete candidate to their left. Other characters -receive no special boundary handling. - -### Resolved mention - -A mention candidate whose exact username resolves to a user under the consuming operation's existing account-status and visibility policy. User-targeted -effects apply to the resolved user ID, not to unresolved source text. - -## Spaces - -### Space - -An instance-scoped collaboration boundary for accepted members and memo placement. A Space is not a tenant, folder, or application-wide authorization -role. - -### Space ID - -The stable internal identity of a Space. It is distinct from the public Space UID and the mutable Space title. - -### Space UID - -The immutable, instance-wide public identifier assigned when a Space is created. It may be user-defined or generated automatically. - -### Space resource name - -The API identity of a Space in the form `spaces/{space UID}`. - -### Space title - -The mutable, non-unique display label of a Space. - -## Tags - -### Tag - -A classification value in a memo tag set, derived from one or more tag occurrences either as a direct tag value or as an implied ancestor. A tag is not a -durable entity and cannot be independently created or renamed. Changing a tag's spelling requires editing the relevant memo source occurrences. - -### Tag introducer - -An ASCII `#` character (U+0023) that is not part of a matched fully-qualified emoji sequence and starts a tag candidate. Visually similar fullwidth and small -number-sign characters are not introducers. - -### Tag occurrence - -An eligible source span in memo Markdown formed by a tag introducer and its consumed source spelling. The spelling emits a tag identifier, which is the -direct tag value. Recognition rules, including excluded Markdown contexts, are defined by the active tag-syntax profile. - -### Eligible tag text - -A contiguous original-source range with no intervening Markdown syntax boundary, classified as textual content by GFM 0.29-gfm or explicitly exposed as -ordinary text by a Memos Markdown extension. Extension nodes are opaque by default, so their source does not produce tags unless the extension definition -opts in. - -### Source spelling - -The exact source substring consumed after the `#` introducer. It includes default-ignorable code points outside matched fully-qualified emoji sequences and -ignored leading combining marks before each segment's starter, both of which the lexer ignores. Inline rendering and source-preserving operations use this -spelling. - -### Direct tag value - -The identifier emitted from one tag occurrence before hierarchy expansion. `book/fiction` is the direct value produced by `#book/fiction`. -Default-ignorable code points outside matched fully-qualified emoji sequences and ignored leading combining marks before each segment's starter are -consumed from the source spelling but omitted from this value. - -### Implied ancestor tag - -A slash-delimited prefix derived from a direct tag value. A direct value of `book/fiction/history` implies the ancestor tags `book` and `book/fiction`. - -### Tag segment - -A non-empty component of a hierarchical tag identifier. `/` separates segments and is consumed only when another non-empty segment follows. A leading slash -produces no tag; a trailing or repeated slash terminates the identifier before that slash. `-`, `+`, and `&` are ordinary segment units: they may appear at -any position, repeat, or form a complete segment. Default-ignorable code points and ignored leading combining marks do not make a segment non-empty. - -### Tag identifier - -The non-empty Unicode code-point sequence emitted from a tag source spelling. The identifier excludes the introducer and all ignored source code points. - -### Display value - -The direct or implied value presented as a derived tag label. Memos does not normalize or case-fold emitted code points for identity, but ignored -default-ignorable code points and ignored leading combining marks are not part of the display value. - -### Comparison key - -The value used for deduplication, counting, filtering, navigation, and exact metadata matching. It is identical to the emitted display value. Two tags -compare as equal only when those emitted Unicode code-point sequences are identical; case and canonical or compatibility-equivalent spellings remain -distinct. Source spellings that differ only by ignored default-ignorable code points or ignored leading combining marks compare equal. Inputs to exact -value filters and metadata lookups are already tag values and are compared as supplied; they are not re-lexed as source spellings. - -### Memo tag set - -The union of direct tag values and their implied ancestor tags for one memo, exposed as `Memo.tags`. It is rebuildable from the memo's Markdown and is not an -authoritative source of tags. Each exactly equal emitted value appears once, even when produced both directly and as an ancestor. Different source -spellings remain separate only when they emit different code-point sequences. - -### Tag metadata rule - -User configuration that selects tag values and supplies presentation or behavior metadata, such as color or content blurring. A metadata rule may match -multiple values and does not create, own, or rename a tag. - -### Tag count - -The number of memo tag sets containing an exactly equal direct or implied tag value, not the number of textual occurrences. A memo containing only -`#book/fiction` contributes one to both the `book` and `book/fiction` counts.