Sync visibility: initiator + not-100%-synced filter — Design

Issue: #395Date: 2026-07-23 Status: Approved for planning

Two related sync-UX gaps from First Pass review:

1. Queue/Sync History should show who initiated each sync.
2. Sets list needs a checkbox to show only sets that are not 100% synced.

---

1. Feature 1 — record who initiated each sync

Problem

SyncJob has no initiator field. Merchants with multiple staff can't tell who kicked off a sync.

Key constraint (verified against Shopify docs, 2026-07-23)

The two auth modes carry different identity:

• Standalone (JWT) — req.user is the full User doc; .email and .name are available.
• Embedded (Shopify session token) — req.shopifyUserId is only the numeric sub claim ("the ID of the user that made the request"). The token contains no name or email (payload is exp, nbf, iss, dest, aud, sub).

Resolving an embedded user to a real name/email requires the StaffMember object, which per Shopify docs "Requires read_users access scope. Also: The app must be a finance embedded app or installed on a Shopify Plus or Advanced store." Most LGS Forge merchants are standard-plan stores, so a staffMember lookup is infeasible for the majority and is explicitly out of scope. We store best-available identity now, structured so a name can be backfilled later if a store ever qualifies.

Data model

New sub-object on syncJobSchema (server/models/SyncJob.js), declared field-by-field (§5.2 — strict mode drops undeclared sub-doc fields silently):

initiatedBy: {
    // 'jwt' = standalone user (email/name known); 'shopify_session' = embedded (only numeric id known)
    source: { type: String, enum: ['jwt', 'shopify_session'] },
    email: String,          // jwt mode
    name: String,           // jwt mode
    shopifyUserId: String   // embedded mode — numeric Shopify staff id as string
}

All fields optional. Absent initiatedBy = system-initiated (e.g. scheduled price_update).

Capture

New helper getInitiator(req) co-located with dualModeAuth (server/middleware/dualModeAuth.js), exported for unit testing:

Auth mode	req signal	Result
Standalone JWT	req.user present	{ source:'jwt', email: req.user.email, name: req.user.name }
Embedded	req.authMode === 'shopify_session_token'	{ source:'shopify_session', shopifyUserId: req.shopifyUserId }
Dev bypass	req.authMode === 'shopify_session_token', no shopifyUserId	{ source:'shopify_session', shopifyUserId: null }

The helper returns undefined when no identity is resolvable, so SyncJob.initiatedBy is simply left unset (→ system).

Threading

Set initiatedBy directly at SyncJob.create(...) inside POST /api/sync (server/routes/api.js), where req is in scope.

Deliberate deviation from the issue text: initiatedBy is not threaded through the BullMQ payload. The SyncJob MongoDB record — the only place the field is persisted or read — is created in the route with req available, so routing the identity through the queue and back would be redundant state. The BullMQ job does not need it.

Also set initiatedBy at the synchronous sealed_add SyncJob creation site(s) so quick-adds attribute correctly. Scheduled/repeatable price_update jobs have no user context → no initiatedBy (renders as "System").

Display

• getInitiatorLabel(job) helper in client/src/utils/jobHelpers.js: job.initiatedBy?.name || job.initiatedBy?.email || (shopifyUserId ? 'Shopify staff #'+shopifyUserId : null) || 'System'.
• SyncHistory.jsx Recent Jobs table: new "Initiated by" column.
• SyncJobDetailModal: show the initiator label.

Testing

• Round-trip schema test (§5.2): create a SyncJob with a full initiatedBy, read it back, assert every sub-field persists. This test fails if any schema line is deleted.
• getInitiator(req) unit test: jwt / embedded / dev-bypass / no-identity branches.
• POST /api/sync sets initiatedBy from req (DI via existing _setDeps patterns / mocked auth).
• getInitiatorLabel unit test covering the fallback chain.

---

2. Feature 2 — "Show only not-100%-synced" filter (all games)

Problem

Merchants syncing an entire catalog want to hide sets that are fully synced and keep working sets — and see a set like SLD resurface when new cards drop into it.

Key constraint (from code)

There is no per-store record in Mongo of which cards a store has synced. ShopifyMTGProduct is a global catalog (no shop field). The only per-store sync signal is Shopify itself. The existing GET /api/synced-sets already derives which sets have a collection from Shopify smart-collection rules (metafield set_code), cached per-shop for 5 minutes (syncedSetsCache).

Precision (approved: approximate is fine for v1)

"% synced" = Shopify collection productsCount ÷ set's expected card count. "Not 100% synced" = productsCount < expected (never synced → 0 → included).

This is approximate: variants-mode syncs, promos, and tokens can skew numerator vs. denominator, so a set may read slightly under/over 100% for benign reasons. It reliably serves both goals (hide finished sets; resurface a set when its card count grows). During implementation the numerator/denominator semantics must be validated against a real synced store (§5.4) — confirm what one productsCount corresponds to under each product mode before trusting the ratio.

Server: sync-count source

Extend the smart-collection query in server/services/shopifyAPI.js. Today getSyncedSetCodes() reads collections(query:"collection_type:smart"){ ... ruleSet.rules{ column condition } } and returns a set of codes. Add productsCount { count } to each collection node and expose:

async getSyncedSetCounts()  // → Map<setCode, productsCount>

getSyncedSetCodes() can be re-derived from the counts map (keys) to keep one source of truth, or retained as-is; either way both must agree.

Cache: store the counts map per-shop with the same 5-min TTL mechanism as syncedSetsCache (extend it or add a sibling), invalidated on the same events (a sync creating a new collection).

Per-game key casing — the join risk (§5.3/§5.4): getSyncedSetCodes() currently .toUpperCase()s the rule condition. Correct for MTG (SetModel.data.code is uppercase) but it would break the join for Pokemon/Riftbound, whose set identifiers are case-preserving slugs. getSyncedSetCounts() must key by the game's actual identifier casing so the map keys are string-equal to the catalog code returned by the list endpoints. Validate against one real smart-collection document per game before finalizing.

Server: filter application

GET /api/sets (MTG) and GET /api/catalog/sets (Pokemon, Riftbound) accept a new onlyIncomplete boolean query param (add to setsQuerySchema and catalogSetsQuerySchema).

When onlyIncomplete=true and the request is authenticated (req.shop present):

1. Load cached synced-counts map for req.shop.
2. Expected card count per set = the same cardCount the list endpoint already displays (so the badge denominator matches the visible card count). Sources, unchanged from today's aggregations:
   • MTG (/sets): physicalCardCountExpr (the projected cardCount).
   • Pokemon (/catalog/sets): PokemonSet.cardCount.
   • Riftbound (/catalog/sets): the computed _cardCount.
3. fullySynced = [codes where productsCount ≥ expected].
4. Add { $match: { <codeField>: { $nin: fullySynced } } } to the aggregation before $sort/$project (§5.6). Server-side array, not a query string → pagination stays correct, no URL-length limit.

/api/catalog/sets uses optionalDualModeAuth; when unauthenticated, onlyIncomplete is a no-op (sync state is unknowable without a shop) — documented in a comment.

Server: response enrichment

Each returned set gains syncedCount and syncedPercent (from the counts map ÷ expected; 0 when not in the map). Powers the client badge and lets the client derive the green "has a collection" check from syncedCount > 0.

Client

client/src/components/SetSelector.jsx:

• Consolidate the separate /synced-sets fetch into the inline syncedCount/syncedPercent now returned by the list endpoints (one source of truth — deliberate deviation approved). The green CheckCircle2 renders when set.syncedCount > 0.
• Add a "Show only not 100% synced" checkbox that sets onlyIncomplete=true on the list request (works for MTG via /sets and other games via /catalog/sets).
• Show a % synced badge next to the set name.
• Client filter is a UX mirror of the server rule; add a comment pointing at the server enforcement (§5.12).

Degradation

If the Shopify smart-collection query fails, the counts map is empty → onlyIncomplete returns all sets and badges hide. Best-effort, never 500 — matches today's /synced-sets behavior.

Per-game parity checklist (§5.1)

• mtg — mirrored: /sets, expected = SetModel size, code casing uppercase.
• pokemon — mirrored: /catalog/sets, expected = PokemonSet.cardCount, code casing = slug (do not uppercase).
• riftbound — mirrored: /catalog/sets, expected = catalog _cardCount; few/no synced collections today so most sets read as incomplete (expected). Code casing = slug.

Testing

• getSyncedSetCounts() parse test against a mocked GraphQL response (ruleSet.rules + productsCount), including the per-game casing assertion.
• onlyIncomplete filter test for /api/sets and /api/catalog/sets per game: cached map + Model mock → asserts fully-synced codes excluded, incomplete/never-synced included, and $match precedes $sort.
• Unauthenticated /catalog/sets with onlyIncomplete → no-op returns all sets.
• Degradation test: count source throws → all sets returned, no badges.
• SetSelector: checkbox toggles the param; % badge and green check render from inline data.

---

Out of scope / follow-ups

• staffMember name/email resolution for embedded users (requires read_users; Plus/Advanced/finance-app only).
• Exact card-level sync diffing (enumerate card_number metafields) — deferred; approximate % chosen for v1.

Quality bars (from CLAUDE.md §7, must all pass)

• npm test + npm run test:client; modified files ≥70% coverage; npm run lint; npm run build (client touched).
• New persisted field declared field-by-field + round-trip test (§5.2).
• Data-matching literals (metafield casing per game) cited from a real document (§5.4).
• Aggregations $project/$match before $sort (§5.6).
• No || 'mtg'-style identity defaults (§5.5).
• Per-game parity stated for mtg/pokemon/riftbound (§5.1).