electronics-inventory/future-plans.md

# Future Plans

## KV store

### Extract into shared library
`kv-store.mjs` is already copied into at least 3 projects (`electronics-inventory`,
`fs-views`, `publication-tool`). Should live in its own Gitea repo as an installable
npm package (`npm install git+https://...`) so changes propagate rather than drift.

### Hierarchical storage structure
The current store is a flat string→value map with prefixed keys (`f:`, `c:`, `bin:`,
etc.) as a manual namespacing convention. This should be replaced with a proper tree:
collections as top-level keys whose values are `Record<id, object>`. Eliminates the
prefix convention, makes collection access direct and self-documenting, and removes
the full-scan `startsWith` pattern from every `list_*` function. Requires a one-time
migration of existing NDJSON data. Best done as part of the shared library rewrite.

### Delta / revision tracking
Add a delta log alongside the main snapshot file (e.g. `inventory.ndjson.deltas`)
that records every `set`/`delete` as a timestamped entry. The main file stays a
clean current-state snapshot; the delta file accumulates the full history. Enables
undo, audit trails, and debugging data corruption.

## Real-time / live updates

### Dual event bus architecture

Two independent buses, bridged by SSE:

**Server bus** (`lib/bus.mjs`, Node `EventEmitter`):
- Route handlers emit a specific event after each mutation (`field:deleted`,
  `bin:changed`, etc.) and nothing else — side effects are not inline
- Effect modules in `server/effects/` subscribe and handle cascading work:
  - `field-effects.mjs` — strips deleted field from all components/bins/bin types
  - `sse-effects.mjs` — broadcasts mutations to connected SSE clients
  - `audit-effects.mjs` — writes delta log (future)
  - `bin-effects.mjs` — e.g. propagates type dimension changes to bins
- New cross-cutting concerns (audit, cache invalidation, notifications) are
  additional listeners — route handlers never grow

**Client bus** (`lib/bus.mjs`, lightweight pub/sub or `EventTarget`):
- `api.mjs` emits on the bus after every successful mutation
- `sse.mjs` (SSE connection client) translates incoming server events to bus emits
- View modules subscribe to relevant events and re-render; they never call each other
- `mock-api.mjs` also emits on the same bus after in-memory mutations, so views
  react correctly in mock mode without any SSE

**SSE bridge**: `sse-effects.mjs` on the server broadcasts to connected clients;
`sse.mjs` on the client receives and re-emits on the client bus. Views are unaware
of whether a change was local or remote.

**Avoiding wildcard listeners**: instead of a wildcard `*` listener (not natively
supported by `EventEmitter`), emit a generic `mutation` event alongside every
specific event. The SSE broadcaster listens to `mutation`; everything else listens
to specific events. New event types are automatically forwarded without touching
the broadcaster.

```js
function emit(event, data) {
    bus.emit(event, data);
    bus.emit('mutation', { event, data });
}
```

**Event granularity**: collection-level events are sufficient (`bins:changed`,
`components:changed`). Passing the affected id or record is optional — views can
use it to do a targeted update or ignore it and re-fetch the collection. Fine-grained
events are an optimisation to add later if full-collection re-fetches become slow.

Ties into the delta tracking plan: `audit-effects.mjs` is another bus listener —
the same mutation path that drives SSE also drives the delta log.

## App architecture

### parse_url mutates too many module-level variables
`parse_url()` directly assigns to a large number of module-level state variables
(`section`, `grid_view_state`, `grid_tab`, `current_grid_id`, `grid_draft`,
`current_panel_idx`, `grid_source_id`, `highlight_cell`, `selected_component_id`).
This is fragile and hard to reason about.

Preferred direction: represent the full UI state as a single immutable state object,
and have `parse_url()` return a new state value rather than mutating globals:
```js
function parse_url(path) {
    return { section, grid_view_state, current_grid_id, ... };
}
state = parse_url(location.pathname); render(state);
```

### render() if/else chain
The render dispatcher is a long chain of bare `else if` branches. Replace with a
lookup table:
```js
const SECTION_RENDERERS = {
    components: render_components,
    inventory:  render_inventory,
    fields:     render_fields,
    grids:      render_grids,
    templates:  render_templates,
};
function render() { sync_nav(); SECTION_RENDERERS[section]?.(); }
```

### UI as a self-contained sub-project

The UI boundary is `api.mjs` — every piece of data the UI touches goes through
named exports in that file. This seam should be made explicit so the UI can be
developed and tested against a mock without a running server.

**Composition root / dependency injection**: `app.mjs` should not import `api.mjs`
directly. Instead it receives the api implementation as a parameter. Two thin entry
files wire it up:

```
main.mjs       — imports real api.mjs, passes to app.start()
mock-main.mjs  — imports mock-api.mjs, passes to app.start()
```

`mock-main.mjs` is a separate deployable (e.g. served at `/mock` or on a dev port),
not a URL flag. The app has no runtime knowledge of which implementation it received.

**mock-api.mjs**: same exports as `api.mjs`, backed by in-memory arrays seeded with
realistic fixture data. Mutations update the in-memory state so the UI behaves
realistically (add/delete/edit all persist within the session). Also emits on the
client bus so cross-view reactivity works identically to the real app. No SSE
connection needed in mock mode — the bus events come from the mock mutations.

**Views never call each other**: once split into modules, `views/bins.mjs` must
not import `views/inventory.mjs`. Cross-section reactions happen exclusively through
the client bus. This is the main structural discipline that makes the split work.

### app.mjs monolith
`app.mjs` is large. Split into per-section view modules (`views/components.mjs`,
`views/grids.mjs`, `views/bins.mjs`, etc.) each owning its local state, subscribing
to bus events at init, and exporting a single `mount(container)` function. The
composition root (`main.mjs`) imports all view modules and registers them.

### Split CSS into per-section files
`style.css` is a single large file and getting hard to navigate. Split into
per-section files (`components.css`, `grids.css`, `bins.css`, etc.) plus a
`base.css` for variables, resets, and shared layout. A `make build` step can
concatenate them into a single `style.css` for deployment, keeping the dev
experience clean without adding a bundler dependency.

### Explicit save in component editor
Currently any change in the component detail panel (linking a file, unlinking an
inventory entry, etc.) is persisted immediately. This makes it hard to experiment or
undo. The component editor dialog should have an explicit Save button and hold all
changes locally until confirmed. Relates to the broader question of whether live
mutations elsewhere in the UI should be deferred similarly.

### DRY / SSoT audit
As the app grows, patterns are being duplicated rather than centralized. Areas to
review:
- Field sorting: same sort-by-name logic appears in both detail view and edit dialog
- Field rendering: `render_field_value()` exists but call sites still sometimes
  inline display logic
- Component display name: `component_display_name()` is the SSoT but there may be
  call sites that still use `c.name` directly
- Server-side: PDF conflict checks, sanitize calls, and rename logic are inline in
  route handlers — could be extracted into a `pdf_service` helper
- General pass to identify and eliminate copy-paste between routes and between
  render functions before the codebase grows further

## Field system

### Improvements

#### Component IDs in dropdowns and lists
The component selector dropdown (e.g. in the inventory entry dialog) only shows the
display name, which is ambiguous when multiple components share a name. Should also
show the component ID.

#### Migrate to integer IDs
Current IDs are timestamp-base36 + random chars. Replace with plain integers
(auto-incrementing). Benefits: human-readable, shorter in URLs, sortable by creation
order, easier to reference verbally.

Migration must be done as an explicit standalone tool (`tools/migrate-ids.mjs` or
similar) that:
1. Reads the current database
2. Builds an old→new ID mapping for all entity types (components, fields, inventory
   entries, grids, PDFs, etc.)
3. Rewrites all references throughout the data (e.g. inventory entries reference
   component IDs, components reference field IDs, file_ids arrays, etc.)
4. Writes a new database file without touching the original until explicitly
   confirmed
5. Keeps a mapping log so the migration is auditable and reversible

Should not be run automatically — operator invokes it deliberately after backing up.

#### Component list sorted by display name
The component left pane list is currently sorted by base name. It should sort by
display name (i.e. the formatter output) so the list order matches what the user
actually sees as the component label.

#### Field display in component detail should use a table
Currently rendered as CSS grid rows but columns don't align because each row is
independent. Use an actual `<table>` so name and value columns line up across all
fields. This is tabular data and a table is the right element.

#### Field value parser chain
Similar to how name formatters use a template chain, field values could be passed
through a parser chain that returns structured data based on field name/type hints.
Examples:

- A field whose name contains `tolerance` could parse `20` as `{ negative: 20, positive: 20 }`
  and `-10/+30` as `{ negative: 10, positive: 30 }`
- URL detection (currently hardcoded in `render_field_value()`) could be one parser
  in this chain rather than a special case
- Mouser/Digi-Key part numbers could be detected and return a structured link target

The parser chain would mirror the template system: user-defined or built-in parsers
keyed by field name pattern, tried in order, returning structured data or `null` to
pass through to the next. `render_field_value()` would then receive parsed data and
render accordingly.

#### Field rendering integrations
With or without a parser chain, `render_field_value()` should gain:
- Mouser/Digi-Key part number fields → auto-craft links to product pages
- More URL-like patterns (without `https://` prefix)

#### Field selector filter
When adding a field to a component in the edit dialog, the dropdown becomes
unwieldy with many fields. Add a filter/search input to the field selector.

#### Custom field input modes
Fields could support multiple named input modes that accept different notations and
convert to the canonical stored value. Example for resistance:
- `direct` — enter `100k` or `4k7` directly
- `3-digit SMD` — enter `104` (decoded as 10×10⁴ = 100kΩ)
- `4-digit SMD` — enter `1003` (decoded as 100×10³ = 100kΩ)

The active input mode is selected via a small dropdown next to the field input, with
a keyboard shortcut to cycle through modes quickly. The last used mode per field is
remembered. Input modes are associated with the field definition (or the measurement
dimension), not per-component. Ties in with the measurement/dimension system —
modes are really just different parsers that produce the same canonical value.

#### Keyboard shortcut for adding a field
When filling out many fields on a component, repeatedly reaching for the mouse to
hit "add field" is slow. Add a configurable keyboard shortcut (e.g. Alt+F) to
focus/trigger the add-field selector from anywhere in the component editor.

#### Search matches field names
The current word-split search only matches field values, not field names. Should
also match on field names so searching `dielectric_characteristics` finds all
components that have that field set, regardless of its value.

#### Parametric search
Allow searching/filtering components by field values, not just names. Examples:
- `resistance < 10k`, `package = 0603`, `voltage_rating >= 50`
- Cross-field queries: find all 0603 resistors under 10kΩ
- Should integrate with the existing word-split search or replace it with a
  richer query language
- Depends on field types (numeric vs string) for range queries to work correctly

### Long term

#### Field grouping / linkage
Some fields naturally belong together (e.g. `frequency_stability` and
`frequency_stability_temp_range`). Options:
- Soft linkage: tag fields with a group name, display grouped in the UI
- Structured fields: a field can be a record type with named sub-fields
  (e.g. `stability: { value: 10, unit: "ppm", temp_low: -40, temp_high: 85 }`)

Structured records are the more powerful option but require a schema system and
more complex UI. Grouping/linkage is a lighter short-term win.

As fields are shared across entity types (components, bins, bin types, and anything
else added later), the field pool grows to span unrelated domains. Groups also serve
as a domain filter in the field selector — when adding a field to a bin type, you
should be able to filter to e.g. "physical" or "storage" fields rather than seeing
electrical component fields mixed in. Each field should be able to belong to one or
more groups.

#### Semantically-aware formatting (acronyms, proper names)
Formatters that apply title case or similar text transformations can corrupt acronyms
(e.g. `NPN` → `Npn`) or brand/proper names. The root cause is that free-text field
values carry no semantic metadata about what kind of string they are. A long-term
fix requires fields to be semantically rich enough that formatters know whether a
value is an acronym, brand name, common noun, number, etc., and apply appropriate
rules per token. Relates to field types and structured field value work.

#### Renderer/parser result cache
Once parsers and formatters run per-render, a cache keyed on field value + template
version would avoid redundant work on large inventories. Invalidated when any
template changes. Not urgent — premature until the parser chain exists.

#### Field types
Currently all field values are free-text strings. Typed fields (numeric,
enum/dropdown) would enable better formatting, validation, and range-aware search.
Prerequisite for parametric search with range operators.

#### Measurement dimensions and unit conversion
Instead of a bare unit string on a field, associate a field with a measurement
dimension (e.g. `temperature`, `resistance`, `frequency`, `voltage`). The dimension
defines the set of valid units and the conversion factors between them (°C, °K, °R,
°F for temperature; Ω, kΩ, MΩ for resistance; etc.).

SI prefixes (k, M, µ, n, p, etc.) are not separate units — they are a presentation
layer on top of a unit. `25kΩ` should be stored as
`{ value: "25", prefix: "k", unit: "Ω" }` — preserving the original string value
and prefix exactly as entered, so no precision or notation is lost.

A canonical numeric form is derived from the stored triple only when needed for
comparison or search queries (e.g. `R < 10k` → compare canonical floats). Display
always reconstructs from the stored `value + prefix + unit`, so `4k7` stays `4k7`
and `25.0` stays `25.0`.

This would allow:
- Lossless storage of entered values (significant digits, notation style preserved)
- Parametric search with cross-prefix comparisons via derived canonical values
- Unit conversion on query (e.g. `temp > 200K` matching a stored `-73°C`)
- Catching unit mismatches at entry time

## Multi-user and access control

### Multi-user support
Currently single-user with no authentication. For shared/team use:
- User accounts with login (session or token-based)
- Per-user audit trail (who added/changed what, ties into delta tracking)
- Optional: user-specific preferences (display units, default grid, etc.)

### Team / permission model
Teams or roles controlling what users can do:
- Read-only members (view inventory, no edits)
- Contributors (add/edit components and inventory)
- Admins (manage fields, grids, users)
- Possible per-resource permissions (e.g. a team owns a specific grid)

### Common user/team library
User and team management is a recurring need across projects. Should be extracted
into a shared library (alongside the planned kv-store library) rather than
reimplemented per project. The library would provide:
- User CRUD with hashed credentials
- Session/token management
- Role and permission primitives
- Middleware for Express (protect routes by role)

The electronics inventory would then depend on this library rather than rolling its
own auth. Other projects (`publication-tool`, future apps) would do the same.

## Deployment

### Read-only public mode
A runtime flag (e.g. `READ_ONLY=1`) that starts the server in a read-only mode
suitable for public-facing deployment:
- All write API endpoints disabled (POST/PUT/DELETE return 403)
- UI hides all edit controls, dialogs, and maintenance actions
- Data served directly from the same `data/` directory

This allows a simple deployment workflow: rsync the `data/` directory from the
private instance to a public server running in read-only mode. No database sync,
no separate export step.

## Editor

### Use CodeMirror 6 for JavaScript input fields
Any field that accepts JavaScript (name formatter templates, future custom search
views, field parsers, etc.) should use a CodeMirror 6 editor instead of a plain
`<textarea>`. Gives syntax highlighting, bracket matching, and a proper editing
experience for JS snippets.

## Template system

### Unified formatter → template pipeline and terminology revision
The current system conflates several distinct concepts under the word "template",
creating ambiguity:

- The HTML `<template>` elements used for UI cloning (internal, not user-facing)
- The user-written JS formatter functions (currently called "templates" in the UI)
- The future idea of user-defined DOM rendering templates

Proposed clearer terminology:
- **Formatter** — a user-written JS function that receives a component and returns a
  structured record (named slots), e.g. `{ label, sublabel, badge, ... }`
- **Renderer** — a DOM fragment template (possibly user-defined) that consumes a
  formatter's record and produces the visual output for a given context (list row,
  detail header, dropdown item, etc.)
- **View template** — the internal HTML `<template>` cloning mechanism (keep as-is,
  but don't expose this term to users)

The pipeline becomes: `component → formatter → record → renderer → DOM`. Formatters
and renderers are decoupled — the same formatter record can feed different renderers
in different contexts. Users can define custom renderers (DOM fragments with named
slot targets) in addition to custom formatters.

This revision also applies to field parsers and search view expressions once those
exist — they all follow the same pattern of JS function → structured output →
context-specific renderer.

## Search & views

### Custom search views
Saved searches defined as JS expressions (similar to the template system), evaluated
against each component to produce a filtered and optionally transformed list. Example
use cases:
- "all components with a non-empty `todo` field"
- "all SMD resistors with no datasheet attached"
- "all components missing a `package` field"

Views would be named, saved, and accessible from the nav or a dedicated views
section. The expression receives the full component object and returns truthy to
include it. Could later be extended to also control sort order and displayed columns.

## Images

### Image gallery / browser
The current image upload for components is minimal. Replace with a proper image
gallery dialog (mirroring the PDF file picker) that shows all uploaded images with
thumbnails and supports:
- File input upload (existing)
- Drag and drop onto the gallery area
- Clipboard paste (Ctrl+V — useful for pasting screenshots directly)
- URL entry (fetch and store server-side)

Images should be manageable from the gallery: rename, delete, link/unlink from
component, open full-size in lightbox. Like the PDF picker, the gallery should be
reusable across components (an image can be shared between components).

## PDF / files

### Auto-select file after upload in file picker
When uploading a PDF from within the file picker (opened from a component), the
newly uploaded file should be automatically linked to that component without
requiring a manual "Select" click.

### File picker search filter
The file picker dialog has no search/filter input. With many datasheets this becomes
unwieldy. Add a filter input at the top of the list that narrows by display name and
filename.

### PDF page count and multi-page navigation
Currently only the first page thumbnail is shown. Could show page count and allow
browsing pages in the lightbox.

## Inventory

### Bins as a storage item type
Support bins (physical containers, boxes, bags, reels, etc.) as inventory items in
their own right — not just as locations. A bin can hold components but is itself a
trackable thing. Bins may contain non-electronic items.

### Inventory type-specific views
Currently the inventory and components views are tightly coupled to the assumption
that everything is an electronic component. Long term, the system should support
multiple item types (components, bins, tools, materials, etc.) with:
- A generic "everything" view showing all inventory regardless of type
- Type-specific views (e.g. the current components view) that filter and present
  items with type-relevant fields and UI
- The current components section becomes one such type-specific view rather than
  the only view

**Implementation approach:** Add a `type` field to items (e.g. `component`, `bin`,
`tool`). Type-specific views are just filtered views over all items. No separate
collection or schema per type — the type field drives which view renders it.

**Migration:** Bulk assignment via the existing field system — e.g. set `type =
"component"` on all current items in one operation, since they're all components.
No per-item manual work needed.

### Inventory URL reflects selected entry
Similar to how components now reflect `/components/:id` in the URL, inventory
entries have no URL state — refreshing loses context.

### Recent locations in inventory entry dialog
When picking a storage location for a component, show a list of recently
used/visited locations at the top so you can quickly re-select where you just were.
Useful when processing a batch of components into the same storage location — you
shouldn't have to navigate the grid picker from scratch each time.

## Bins

### Bin types
Define reusable bin type records (e.g. "Sortimo L-Boxx insert small", "Wago 221
connector box") that store physical dimensions (mm), and optionally a default
compartment layout. When creating or editing a bin, the user picks a type and the
dimensions are pre-filled — no need to re-enter for every bin of the same model.
This also enables filtering/grouping bins by type, and makes it easy to re-process
all bins of a type if the corner algorithm improves.

### Generic fields on bins and bin types
Bins and bin types should both support the same generic field system as components —
arbitrary key/value pairs from the shared field definitions. Examples: color, material,
manufacturer, max load, purchase link. Bin types carry the "template" fields (e.g.
nominal dimensions from the datasheet) while individual bins carry instance-specific
fields (e.g. actual color of that specific unit).

Because fields are shared across components, bins, and anything else that grows into
the system, they will quickly span unrelated domains. Field grouping (see Field system
section) becomes important here so the field selector can be filtered to show only
relevant fields for the current entity type.

### Duplicate any entity
All objects in the system should be duplicatable: components, bin types, bins, grids,
templates, inventory entries, and eventually source images. The duplicate operation
creates a new record with all fields copied, then opens it in an edit dialog so the
user can adjust what differs. Bin type duplication is especially common — same
physical container model in different colors or configurations. Source images are a
later case since they reference uploaded files; duplication there would mean creating
a new metadata record pointing to the same underlying file (or an explicit copy).

## Grids

### Grid view layers
Allow a storage grid to reference one or more separate "view layer" grids that share
the same logical layout but use different source images. Example: the storage grid
uses close-up photos of individual cells for identification, while a view layer uses
a wider photo of the lid or top side for orientation.

Key design points:
- Grids get a classification: `storage` (can hold inventory) or `view` (display
  only, referenced by storage grids)
- View layers may use a different panel sub-grid layout (fewer, larger panels) as
  long as the final logical row×col count matches the storage grid
- In the grid viewer, layers can be toggled to switch between the storage view and
  any attached view layers
- A storage grid can have multiple view layers (e.g. lid photo, tray photo, labeled
  overlay)

### Irregular grid layouts and merged cells
Real storage boxes rarely have perfectly uniform grids. Two distinct physical
configurations need to be supported:

**Type A — uniform grid with merged cells:** A regular N×M grid where some adjacent
cells are physically merged into one larger cell (always an integer multiple of the
base cell size). Common in component assortment boxes. A merged cell is both a
physical and logical unit — you store one thing in it.

**Type B — stacked sub-grids:** A container where each row (or section) has a
different column count and cell size. Example: 5 rows of 5 small columns, then 1
row of 4 medium columns, then 1 row with a single large drawer. Cells are not
multiples of a common base — the sections are structurally independent.

**Logical merging (cell groups):** Independent of physical layout, a user should be
able to group several cells into a single named logical location. The motivating
case is a batch of 50 components that won't fit in one cell — they spill across 3
cells, but you want one inventory entry saying "these cells together hold this
batch", not three separate entries to keep in sync. This is purely a
storage/inventory concern, not a grid layout concern.

**Open question — architecture:** Should this be:
1. A single generic nested/hierarchical grid model flexible enough to encode both
   types (more complex but unified), or
2. Two explicit grid styles (`uniform+merges` and `stacked-sections`) that cover
   the common cases without a fully general solution?

Option 2 is likely sufficient for real-world boxes and much easier to implement and
display. Worth prototyping before committing to a generic model.

### Multi-cell grid storage selection
A component stored in a grid should be able to span multiple cells, since larger
parts often occupy more than one cell. The graphical cell picker in the inventory
dialog should support selecting a range or set of cells rather than a single cell.
The grid viewer should reflect multi-cell occupancy in its count badges and cell
highlighting.

### Grid URL state
Navigating into a grid viewer updates the URL correctly, but the grid list and
draft state have no URL representation.