# Future Plans ## KV store ### Extract into shared library `kv-store.mjs` is already copied into at least 3 projects (`electronics-inventory`, `fs-views`, `publication-tool`). Should live in its own Gitea repo as an installable npm package (`npm install git+https://...`) so changes propagate rather than drift. ### Hierarchical storage structure The current store is a flat string→value map with prefixed keys (`f:`, `c:`, `bin:`, etc.) as a manual namespacing convention. This should be replaced with a proper tree: collections as top-level keys whose values are `Record`. Eliminates the prefix convention, makes collection access direct and self-documenting, and removes the full-scan `startsWith` pattern from every `list_*` function. Requires a one-time migration of existing NDJSON data. Best done as part of the shared library rewrite. ### Delta / revision tracking Add a delta log alongside the main snapshot file (e.g. `inventory.ndjson.deltas`) that records every `set`/`delete` as a timestamped entry. The main file stays a clean current-state snapshot; the delta file accumulates the full history. Enables undo, audit trails, and debugging data corruption. ## Real-time / live updates ### Dual event bus architecture Two independent buses, bridged by SSE: **Server bus** (`lib/bus.mjs`, Node `EventEmitter`): - Route handlers emit a specific event after each mutation (`field:deleted`, `bin:changed`, etc.) and nothing else — side effects are not inline - Effect modules in `server/effects/` subscribe and handle cascading work: - `field-effects.mjs` — strips deleted field from all components/bins/bin types - `sse-effects.mjs` — broadcasts mutations to connected SSE clients - `audit-effects.mjs` — writes delta log (future) - `bin-effects.mjs` — e.g. propagates type dimension changes to bins - New cross-cutting concerns (audit, cache invalidation, notifications) are additional listeners — route handlers never grow **Client bus** (`lib/bus.mjs`, lightweight pub/sub or `EventTarget`): - `api.mjs` emits on the bus after every successful mutation - `sse.mjs` (SSE connection client) translates incoming server events to bus emits - View modules subscribe to relevant events and re-render; they never call each other - `mock-api.mjs` also emits on the same bus after in-memory mutations, so views react correctly in mock mode without any SSE **SSE bridge**: `sse-effects.mjs` on the server broadcasts to connected clients; `sse.mjs` on the client receives and re-emits on the client bus. Views are unaware of whether a change was local or remote. **Avoiding wildcard listeners**: instead of a wildcard `*` listener (not natively supported by `EventEmitter`), emit a generic `mutation` event alongside every specific event. The SSE broadcaster listens to `mutation`; everything else listens to specific events. New event types are automatically forwarded without touching the broadcaster. ```js function emit(event, data) { bus.emit(event, data); bus.emit('mutation', { event, data }); } ``` **Event granularity**: collection-level events are sufficient (`bins:changed`, `components:changed`). Passing the affected id or record is optional — views can use it to do a targeted update or ignore it and re-fetch the collection. Fine-grained events are an optimisation to add later if full-collection re-fetches become slow. Ties into the delta tracking plan: `audit-effects.mjs` is another bus listener — the same mutation path that drives SSE also drives the delta log. ## App architecture ### parse_url mutates too many module-level variables `parse_url()` directly assigns to a large number of module-level state variables (`section`, `grid_view_state`, `grid_tab`, `current_grid_id`, `grid_draft`, `current_panel_idx`, `grid_source_id`, `highlight_cell`, `selected_component_id`). This is fragile and hard to reason about. Preferred direction: represent the full UI state as a single immutable state object, and have `parse_url()` return a new state value rather than mutating globals: ```js function parse_url(path) { return { section, grid_view_state, current_grid_id, ... }; } state = parse_url(location.pathname); render(state); ``` ### render() if/else chain The render dispatcher is a long chain of bare `else if` branches. Replace with a lookup table: ```js const SECTION_RENDERERS = { components: render_components, inventory: render_inventory, fields: render_fields, grids: render_grids, templates: render_templates, }; function render() { sync_nav(); SECTION_RENDERERS[section]?.(); } ``` ### UI as a self-contained sub-project The UI boundary is `api.mjs` — every piece of data the UI touches goes through named exports in that file. This seam should be made explicit so the UI can be developed and tested against a mock without a running server. **Composition root / dependency injection**: `app.mjs` should not import `api.mjs` directly. Instead it receives the api implementation as a parameter. Two thin entry files wire it up: ``` main.mjs — imports real api.mjs, passes to app.start() mock-main.mjs — imports mock-api.mjs, passes to app.start() ``` `mock-main.mjs` is a separate deployable (e.g. served at `/mock` or on a dev port), not a URL flag. The app has no runtime knowledge of which implementation it received. **mock-api.mjs**: same exports as `api.mjs`, backed by in-memory arrays seeded with realistic fixture data. Mutations update the in-memory state so the UI behaves realistically (add/delete/edit all persist within the session). Also emits on the client bus so cross-view reactivity works identically to the real app. No SSE connection needed in mock mode — the bus events come from the mock mutations. **Views never call each other**: once split into modules, `views/bins.mjs` must not import `views/inventory.mjs`. Cross-section reactions happen exclusively through the client bus. This is the main structural discipline that makes the split work. ### app.mjs monolith `app.mjs` is large. Split into per-section view modules (`views/components.mjs`, `views/grids.mjs`, `views/bins.mjs`, etc.) each owning its local state, subscribing to bus events at init, and exporting a single `mount(container)` function. The composition root (`main.mjs`) imports all view modules and registers them. ### Split CSS into per-section files `style.css` is a single large file and getting hard to navigate. Split into per-section files (`components.css`, `grids.css`, `bins.css`, etc.) plus a `base.css` for variables, resets, and shared layout. A `make build` step can concatenate them into a single `style.css` for deployment, keeping the dev experience clean without adding a bundler dependency. ### Explicit save in component editor Currently any change in the component detail panel (linking a file, unlinking an inventory entry, etc.) is persisted immediately. This makes it hard to experiment or undo. The component editor dialog should have an explicit Save button and hold all changes locally until confirmed. Relates to the broader question of whether live mutations elsewhere in the UI should be deferred similarly. ### DRY / SSoT audit As the app grows, patterns are being duplicated rather than centralized. Areas to review: - Field sorting: same sort-by-name logic appears in both detail view and edit dialog - Field rendering: `render_field_value()` exists but call sites still sometimes inline display logic - Component display name: `component_display_name()` is the SSoT but there may be call sites that still use `c.name` directly - Server-side: PDF conflict checks, sanitize calls, and rename logic are inline in route handlers — could be extracted into a `pdf_service` helper - General pass to identify and eliminate copy-paste between routes and between render functions before the codebase grows further ## Field system ### Improvements #### Component IDs in dropdowns and lists The component selector dropdown (e.g. in the inventory entry dialog) only shows the display name, which is ambiguous when multiple components share a name. Should also show the component ID. #### Migrate to integer IDs Current IDs are timestamp-base36 + random chars. Replace with plain integers (auto-incrementing). Benefits: human-readable, shorter in URLs, sortable by creation order, easier to reference verbally. Migration must be done as an explicit standalone tool (`tools/migrate-ids.mjs` or similar) that: 1. Reads the current database 2. Builds an old→new ID mapping for all entity types (components, fields, inventory entries, grids, PDFs, etc.) 3. Rewrites all references throughout the data (e.g. inventory entries reference component IDs, components reference field IDs, file_ids arrays, etc.) 4. Writes a new database file without touching the original until explicitly confirmed 5. Keeps a mapping log so the migration is auditable and reversible Should not be run automatically — operator invokes it deliberately after backing up. #### Component list sorted by display name The component left pane list is currently sorted by base name. It should sort by display name (i.e. the formatter output) so the list order matches what the user actually sees as the component label. #### Field display in component detail should use a table Currently rendered as CSS grid rows but columns don't align because each row is independent. Use an actual `` so name and value columns line up across all fields. This is tabular data and a table is the right element. #### Field value parser chain Similar to how name formatters use a template chain, field values could be passed through a parser chain that returns structured data based on field name/type hints. Examples: - A field whose name contains `tolerance` could parse `20` as `{ negative: 20, positive: 20 }` and `-10/+30` as `{ negative: 10, positive: 30 }` - URL detection (currently hardcoded in `render_field_value()`) could be one parser in this chain rather than a special case - Mouser/Digi-Key part numbers could be detected and return a structured link target The parser chain would mirror the template system: user-defined or built-in parsers keyed by field name pattern, tried in order, returning structured data or `null` to pass through to the next. `render_field_value()` would then receive parsed data and render accordingly. #### Field rendering integrations With or without a parser chain, `render_field_value()` should gain: - Mouser/Digi-Key part number fields → auto-craft links to product pages - More URL-like patterns (without `https://` prefix) #### Field selector filter When adding a field to a component in the edit dialog, the dropdown becomes unwieldy with many fields. Add a filter/search input to the field selector. #### Custom field input modes Fields could support multiple named input modes that accept different notations and convert to the canonical stored value. Example for resistance: - `direct` — enter `100k` or `4k7` directly - `3-digit SMD` — enter `104` (decoded as 10×10⁴ = 100kΩ) - `4-digit SMD` — enter `1003` (decoded as 100×10³ = 100kΩ) The active input mode is selected via a small dropdown next to the field input, with a keyboard shortcut to cycle through modes quickly. The last used mode per field is remembered. Input modes are associated with the field definition (or the measurement dimension), not per-component. Ties in with the measurement/dimension system — modes are really just different parsers that produce the same canonical value. #### Keyboard shortcut for adding a field When filling out many fields on a component, repeatedly reaching for the mouse to hit "add field" is slow. Add a configurable keyboard shortcut (e.g. Alt+F) to focus/trigger the add-field selector from anywhere in the component editor. #### Search matches field names The current word-split search only matches field values, not field names. Should also match on field names so searching `dielectric_characteristics` finds all components that have that field set, regardless of its value. #### Parametric search Allow searching/filtering components by field values, not just names. Examples: - `resistance < 10k`, `package = 0603`, `voltage_rating >= 50` - Cross-field queries: find all 0603 resistors under 10kΩ - Should integrate with the existing word-split search or replace it with a richer query language - Depends on field types (numeric vs string) for range queries to work correctly ### Long term #### Field grouping / linkage Some fields naturally belong together (e.g. `frequency_stability` and `frequency_stability_temp_range`). Options: - Soft linkage: tag fields with a group name, display grouped in the UI - Structured fields: a field can be a record type with named sub-fields (e.g. `stability: { value: 10, unit: "ppm", temp_low: -40, temp_high: 85 }`) Structured records are the more powerful option but require a schema system and more complex UI. Grouping/linkage is a lighter short-term win. As fields are shared across entity types (components, bins, bin types, and anything else added later), the field pool grows to span unrelated domains. Groups also serve as a domain filter in the field selector — when adding a field to a bin type, you should be able to filter to e.g. "physical" or "storage" fields rather than seeing electrical component fields mixed in. Each field should be able to belong to one or more groups. #### Semantically-aware formatting (acronyms, proper names) Formatters that apply title case or similar text transformations can corrupt acronyms (e.g. `NPN` → `Npn`) or brand/proper names. The root cause is that free-text field values carry no semantic metadata about what kind of string they are. A long-term fix requires fields to be semantically rich enough that formatters know whether a value is an acronym, brand name, common noun, number, etc., and apply appropriate rules per token. Relates to field types and structured field value work. #### Renderer/parser result cache Once parsers and formatters run per-render, a cache keyed on field value + template version would avoid redundant work on large inventories. Invalidated when any template changes. Not urgent — premature until the parser chain exists. #### Field types Currently all field values are free-text strings. Typed fields (numeric, enum/dropdown) would enable better formatting, validation, and range-aware search. Prerequisite for parametric search with range operators. #### Measurement dimensions and unit conversion Instead of a bare unit string on a field, associate a field with a measurement dimension (e.g. `temperature`, `resistance`, `frequency`, `voltage`). The dimension defines the set of valid units and the conversion factors between them (°C, °K, °R, °F for temperature; Ω, kΩ, MΩ for resistance; etc.). SI prefixes (k, M, µ, n, p, etc.) are not separate units — they are a presentation layer on top of a unit. `25kΩ` should be stored as `{ value: "25", prefix: "k", unit: "Ω" }` — preserving the original string value and prefix exactly as entered, so no precision or notation is lost. A canonical numeric form is derived from the stored triple only when needed for comparison or search queries (e.g. `R < 10k` → compare canonical floats). Display always reconstructs from the stored `value + prefix + unit`, so `4k7` stays `4k7` and `25.0` stays `25.0`. This would allow: - Lossless storage of entered values (significant digits, notation style preserved) - Parametric search with cross-prefix comparisons via derived canonical values - Unit conversion on query (e.g. `temp > 200K` matching a stored `-73°C`) - Catching unit mismatches at entry time ## Multi-user and access control ### Multi-user support Currently single-user with no authentication. For shared/team use: - User accounts with login (session or token-based) - Per-user audit trail (who added/changed what, ties into delta tracking) - Optional: user-specific preferences (display units, default grid, etc.) ### Team / permission model Teams or roles controlling what users can do: - Read-only members (view inventory, no edits) - Contributors (add/edit components and inventory) - Admins (manage fields, grids, users) - Possible per-resource permissions (e.g. a team owns a specific grid) ### Common user/team library User and team management is a recurring need across projects. Should be extracted into a shared library (alongside the planned kv-store library) rather than reimplemented per project. The library would provide: - User CRUD with hashed credentials - Session/token management - Role and permission primitives - Middleware for Express (protect routes by role) The electronics inventory would then depend on this library rather than rolling its own auth. Other projects (`publication-tool`, future apps) would do the same. ## Deployment ### Read-only public mode A runtime flag (e.g. `READ_ONLY=1`) that starts the server in a read-only mode suitable for public-facing deployment: - All write API endpoints disabled (POST/PUT/DELETE return 403) - UI hides all edit controls, dialogs, and maintenance actions - Data served directly from the same `data/` directory This allows a simple deployment workflow: rsync the `data/` directory from the private instance to a public server running in read-only mode. No database sync, no separate export step. ## Editor ### Use CodeMirror 6 for JavaScript input fields Any field that accepts JavaScript (name formatter templates, future custom search views, field parsers, etc.) should use a CodeMirror 6 editor instead of a plain `