Commit Graph

250 Commits

Author SHA1 Message Date
Garrett Allen fa2dcf59d5 fix: upgrade golang.org/x/net, grpc and Go to 1.27.2 (CVE-2026-78669, CVE-2026-84445) (#6453)
Co-authored-by: johnnyjoygh <johnnyjoygg@gmail.com>
2026-10-11 09:20:42 +08:00
johnnyjoygh f32d367c7f chore: update database drivers
Drop the D1 store driver, its migrations, emulator, design doc, filter
dialect, and test-harness wiring. Memos supports SQLite, MySQL, and
PostgreSQL only.
2026-09-17 09:31:42 +08:00
bluedbird 471745b431 fix: harden attachment, auth, and import paths for public deployment
Bound work an anonymous or low-privilege caller can force:

- Bound thumbnail decoding by declared pixel count before decoding and
  remember images that cannot be thumbnailed (over the bound or not
  decodable), so a small image declaring billions of pixels is served
  as-is instead of decoded on every request. Transient storage read
  errors are not remembered. The bound lives in internal/imagelimit and
  is shared with EXIF stripping.
- Cap request bodies at 16 MiB for procedures that carry no file bytes;
  only inline attachment creation, transcription, and chunked uploads
  keep their larger caps.
- Digest over-long submitted usernames before using them as sign-in
  limiter keys, add header and idle timeouts to the HTTP server, limit
  filter expression size, and cap the SQLite regexp cache.

Close correctness gaps:

- Delete the thumbnail the file server actually writes (.v2.jpeg), its
  failure marker, and the legacy name when an attachment is removed.
- Decide EXIF stripping from the sniffed content as well as the declared
  type, so a mislabeled JPEG keeps no metadata.
- Revoke every other session when a password changes; an admin reset
  revokes all of the target's sessions. Prune expired refresh tokens.
- Enforce the 2 MiB avatar limit the web client already applies.
- Keep S3 object keys unique when the path template has no {uuid}.
- Reject negative attachment page tokens.
- Charge memo archive imports against the write and upload budgets,
  charge repeated validate-only plans, discard only attachments an
  import actually stored, and refuse archives that declare far more
  bytes than they hold.
- Load database-stored attachment bytes only after authorization.
- Remove the unused, unguarded linkmeta.GetImage fetcher.

Frontend: send no referrer when loading external images in memos and
link previews, and drop picture/source from the sanitizer so every
remote image goes through the https-only src rule.

Dependencies: grpc 1.83.1 and x/crypto 0.56.0 for reported advisories.
2026-09-17 08:47:44 +08:00
bluedbird 9ca12926c7 fix(d1): close guard gaps, chunk-free list bindings, and statement
layout

Review follow-up for the D1 driver.

- LATEST.sql kept two index statements in one paragraph; the bridge
  sends a
  paragraph as one prepared statement, which D1 refuses. Separate them,
  and
  join buffered scripts in the sql adapter with a blank line so several
  migration files commit over the bridge.
- List lookups by id, uid, and username bound one parameter per value
  and
  failed above D1's 100-parameter limit for pages larger than 100 rows.
  Bind
  the whole list as one JSON array through json_each instead, which also
  replaces the uid resolution threshold and the chunk-and-resort paths.
- Write-policy validation now returns the memo state it read, and every
  policy-checked write guards on exactly that state: memo update, memo
  mutation, share creation, and attachment creation, update, and
  deletion,
  including the expected memo content and attachment bindings. Memo,
  user,
  and space deletion guard their pre-read row sets on count plus highest
  id
  so a row swapped for a newer one is caught.
- A 429 is no longer retried; the API rate limit that produces it blocks
  the
  caller for minutes.
- The reference Worker defaults a null result set to an empty array, and
  the
  comments claiming a batch never reads its own writes are corrected.

The emulator now judges statement boundaries with sqlite3_complete: a
bridge
entry or a parameterized REST entry holding two statements is refused
with
D1's message, and a parameter-free REST entry runs as a script. New
tests
interpose on the transport to change the database between a method's
reads
and its batch, apply the real schema over both transports, pin the
one-statement rule, and list by 250-entry id lists on every driver.
2026-09-16 23:09:28 +08:00
boojack c506f67d22 feat(server): rate limit sign-in, signup, fetches, uploads, and writes (#6315) 2026-09-13 11:58:26 +08:00
boojack 751005bc19 feat(user): make email an optional, canonical, instance-unique attribute (#6314) 2026-09-12 12:21:41 +08:00
boojack 5b6d8f87bf refactor(server): remove RSS feed support (#6243) 2026-08-28 23:06:25 +08:00
amblued 57a4b7aee6 chore(go): upgrade to Go 1.27 (#6217) 2026-08-20 23:37:02 +08:00
amblued 38412eb75a fix(storage): proxy S3 attachments through authenticated routes (#6210) 2026-08-18 23:38:35 +08:00
Johnny f3a6bfa8d9 chore(deps): update backend dependencies (#6199) 2026-08-17 09:17:58 +08:00
Johnny 7d971b1c61 feat(storage): add named attachment storage drivers (#6184) 2026-08-13 22:37:11 +08:00
Anupam Mediratta 3b8d2558ed fix: upgrade golang.org/x/crypto to 0.52.0 (CVE-2026-39829) (#6152) 2026-08-06 15:25:04 +08:00
boojack f0e4a5624f feat(filter): expand CEL filter surface with startsWith/endsWith, matches(), and all()
Let users write three more CEL constructs in the filter field, each compiled to
SQL across SQLite/MySQL/Postgres:

- Scalar startsWith()/endsWith() on content/filename/mime_type (case-insensitive)
- matches() regex: PG ~, MySQL/SQLite REGEXP (Go-backed SQLite fn), validated at
  compile time via cel.ValidateRegexLiterals()
- all() comprehension over tags via per-element subqueries, non-empty required

Also: contains() now escapes LIKE metacharacters (%, _, \); cross-dialect render
tests plus behavioral tests; cel-go bumped to v0.28.1; new operators surfaced in
the frontend shortcut guide.
2026-06-15 23:22:28 +08:00
boojack 777d227eb9 feat: add OpenAPI-driven MCP support (#6026) 2026-06-09 09:16:50 +08:00
boojack 2a4638b332 chore: remove MCP server 2026-06-05 08:38:57 +08:00
boojack 5ccba98adc refactor: split STT and Audio-LLM into separate interfaces (#5928) 2026-05-03 00:21:58 +08:00
boojack 616487fa37 chore: update backend and frontend dependencies (#5900)
Co-authored-by: memoclaw <265580040+memoclaw@users.noreply.github.com>
2026-04-28 08:25:26 +08:00
boojack a7fd1dacc9 refactor(ai): use official provider SDKs (#5845) 2026-04-16 22:27:57 +08:00
boojack 330291d4d9 feat(user): add per-user tag metadata settings (#5735) 2026-03-18 23:15:14 +08:00
memoclaw 12e2205cb6 chore(backend): update Go toolchain and dependencies (#5730) 2026-03-16 21:07:52 +08:00
Steven 47d9414702 feat: add MCP server with PAT authentication
Embeds a Model Context Protocol (MCP) server into the Memos HTTP
process, exposing memo operations as MCP tools at POST/GET /mcp using
Streamable HTTP transport.

Authentication is PAT-only — requests without a valid personal access
token receive HTTP 401. Six tools are exposed: list_memos, get_memo,
create_memo, update_memo, delete_memo, and search_memos, all scoped to the authenticated user.
2026-02-24 22:54:51 +08:00
Steven 71e8a06463 chore: upgrade Echo v4 to v5.0.3 2026-02-10 09:15:27 +08:00
Steven 8770b186e4 fix: add Unicode case-insensitive search for SQLite (#5559)
Add custom memos_unicode_lower() SQLite function to enable proper
case-insensitive text search for non-English languages (Cyrillic,
Greek, CJK, etc.).

Previously, SQLite's LOWER() only worked for ASCII characters due to
modernc.org/sqlite lacking ICU extension. This caused searches for
non-English text to be case-sensitive (e.g., searching 'блины' wouldn't
find 'Блины').

Changes:
- Add store/db/sqlite/functions.go with Unicode case folding function
- Register custom function using golang.org/x/text/cases.Fold()
- Update filter renderer to use custom function for SQLite dialect
- Add test for Unicode case-insensitive search
- Make golang.org/x/text a direct dependency

Fixes #5559
2026-02-02 21:10:07 +08:00
Johnny f4cf2e9559 test: improve migration tests stability and maintainability
- Fix linting issues and address testcontainers deprecation in store/test/containers.go
- Extract MemosStartupWaitStrategy for consistent container health checks
- Refactor migrator_test.go to use a consolidated testMigration helper, reducing duplication
- Add store/test/Dockerfile for optimized local test image builds
2026-01-07 23:04:12 +08:00
Johnny bd02de9895 chore: add store tests (#5397) 2025-12-31 21:26:35 +08:00
Johnny 09afa579e4 chore: implement session sliding expiration and JWT authentication
- Added UpdateSessionLastAccessed method to update session access time.
- Enhanced Authenticate method to support both session cookie and JWT token authentication.
- Introduced AuthResult struct to encapsulate authentication results.
- Added SetUserInContext function to simplify context management for authenticated users.

refactor(auth): streamline gRPC and HTTP authentication

- Removed gRPC authentication interceptor and replaced it with a unified approach using GatewayAuthMiddleware for HTTP requests.
- Updated Connect interceptors to utilize the new authentication logic.
- Consolidated public and admin-only method checks into service layer for better maintainability.

chore(api): clean up unused code and improve documentation

- Removed deprecated logger interceptor and unused gRPC server code.
- Updated ACL configuration documentation for clarity on public and admin-only methods.
- Enhanced metadata handling in Connect RPC to ensure consistent header access.

fix(server): simplify server startup and shutdown process

- Eliminated cmux dependency for handling HTTP and gRPC traffic.
- Streamlined server initialization and shutdown logic for better performance and readability.
2025-12-15 10:04:11 +08:00
Johnny edd3ced9bf refactor: migrate to connect-rpc (#5338) 2025-12-11 19:49:07 +08:00
Steven 8f29db2f49 fix(server): prevent memory exhaustion in thumbnail generation
Address high memory usage when opening resource tab (fixes #5183) by implementing:

1. Concurrency control: Limit thumbnail generation to 3 concurrent operations using semaphore to prevent memory exhaustion when many images are requested simultaneously

2. S3 optimization: Skip server-side thumbnail generation for S3-stored images by default. S3 images now use presigned URLs directly, avoiding:
   - Downloading large images from S3 into server memory
   - Decoding and resizing images on the server
   - High memory consumption during batch requests

3. Memory management improvements:
   - Explicitly clear blob and decoded image from memory after use
   - Restructure thumbnail cache check to avoid unnecessary semaphore acquisition
   - Double-check pattern to prevent duplicate generation while waiting

This restores the original S3 behavior before commit e4f6345 while maintaining thumbnail support for local/database storage.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-07 09:04:21 +08:00
Claude 739fd2cde6 refactor: update markdown parser
- Removed the `nodes` field from the `Memo` interface in `memo_service.ts`.
- Updated the `createBaseMemo` function and the `Memo` message functions to reflect the removal of `nodes`.
- Cleaned up the serialization and deserialization logic accordingly.

chore: remove code-inspector-plugin from Vite configuration

- Deleted the `codeInspectorPlugin` from the Vite configuration in `vite.config.mts`.
- Simplified the plugins array to include only `react` and `tailwindcss`.
2025-10-26 11:28:40 +08:00
Steven d8f7a4e739 chore: tweak demo data 2025-10-21 23:41:46 +08:00
Steven 81e852e3b5 chore: update gomark dependency 2025-10-21 23:09:27 +08:00
dependabot[bot] 3514180721 chore: bump golang.org/x/crypto from 0.41.0 to 0.42.0 (#5141)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-08 09:30:37 +08:00
dependabot[bot] 0f19713fce chore: bump github.com/aws/aws-sdk-go-v2/config from 1.31.6 to 1.31.12 (#5139)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-08 09:28:25 +08:00
dependabot[bot] ca62b8cd0c chore: bump google.golang.org/protobuf from 1.36.8 to 1.36.9 (#5140)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-08 09:28:17 +08:00
dependabot[bot] cca33af8fd chore: bump google.golang.org/grpc from 1.75.0 to 1.75.1 (#5142)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-08 09:28:06 +08:00
dependabot[bot] 763a0d0dea chore: bump golang.org/x/mod from 0.27.0 to 0.28.0 (#5143)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-08 09:27:58 +08:00
github-actions[bot] 074c580214 chore: bump gomark
- Bump protoc-gen-go version from v1.36.8 to v1.36.9 in multiple generated files.
- Enhance OpenAPI descriptions for various endpoints to improve clarity and readability.
- Fix typo in the description list handling in the markdown service.
- Update HTMLElement component to handle children and self-closing tags correctly.
- Modify types in the markdown service to include children and self-closing properties for HTML elements.
2025-09-28 16:15:55 +08:00
Steven 5ad2038b1a feat: update gomark dependency and refactor markdown parsing logic 2025-09-17 21:09:30 +08:00
dependabot[bot] 1954eeb37b chore: bump github.com/spf13/cobra from 0.0.3 to 1.10.1 (#5059)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-02 22:21:21 +08:00
Johnny 3a7538f787 chore: fix linter 2025-08-31 20:29:40 +08:00
Johnny a373673087 chore: go mod tidy 2025-08-31 20:24:18 +08:00
Johnny 9557150a87 chore: upgrade go version to 1.25 2025-08-31 19:45:41 +08:00
Johnny e427344e82 chore: upgrade dependencies 2025-08-31 13:14:10 +08:00
dependabot[bot] 07e270564a chore: bump golang.org/x/net from 0.40.0 to 0.42.0 (#4948)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-03 19:05:49 +08:00
dependabot[bot] 94b58f0691 chore: bump google.golang.org/grpc from 1.72.2 to 1.74.2 (#4949)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-03 19:05:40 +08:00
dependabot[bot] 5a7693c90a chore: bump github.com/google/cel-go from 0.25.0 to 0.26.0 (#4951)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-03 19:05:30 +08:00
dependabot[bot] 11685c1f4d chore: bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.17.82 to 1.18.2 (#4952)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-08-03 19:05:22 +08:00
dependabot[bot] d60aee7823 chore: bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.80.0 to 1.82.0 (#4806)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-07-05 19:32:25 +08:00
dependabot[bot] 3c512939a8 chore: bump github.com/go-sql-driver/mysql from 1.9.2 to 1.9.3 (#4807)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-07-05 11:27:57 +08:00
dependabot[bot] 17cd19588d chore: bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.17.77 to 1.17.82 (#4808)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-07-03 22:34:19 +08:00