Appearance
Making a new TCG pluggable — design
Date: 2026-09-01 Status: Proposal. Not implemented. Architectural → propose-then-wait (CLAUDE.md §3). Baseline: PR #686 / commit 7330cd5 (claude/add-lorcana-catalog-and-sync), the 4th game. Question: what has to change so that going from 4 games to ~100 doesn't cost 100× the code?
0. TL;DR
- The duplication is real and larger than it looks. After erasing the game name and comments,
LorcanaPrice.jsandRiftboundPrice.jsare byte-for-byte identical;LorcanaSet.js/RiftboundSet.jslikewise;LorcanaProduct.jsdiffers by one line;ShopifyLorcanaProductVariant.jsby four. The two plugins' 32 shared methods are 90% line-identical, with 20 of them at 100%. - The tests are not duplicated — the prior analysis had this backwards. The eight Lorcana test files are only 20–34% identical to their Riftbound counterparts despite having the same test count (86 vs 85). They are structurally parallel and textually distinct because the assertions carry real per-game upstream literals. That is §5.4 working as designed, and a refactor must preserve it rather than delete it.
- The single highest value-per-risk change is not an abstraction at all: it is deleting hand-maintained game lists that the plugin registry can already answer.
client/src/lib/rarityOptions.jsis 100% derivable fromplugin.raritiesfor all four games — verified, all 47 rows. That work carries no §5.9 risk because it adds no seam. - The manifest is the right end state, but should be discovered, not designed. Extract what is provably identical today (a
TcgcsvGamePluginbase class, model factories, one shared importer), then let game #5 tell us what the remaining constructor arguments are. Writing a manifest schema now, from two samples, is exactly §5.9. - Full end state would have taken PR #686 from 36 files / 4,631 code lines to roughly 5 files / ~760 lines. The near-term steps alone (no manifest, no shared importer) would have saved about 480.
- One live bug found while measuring:
client/src/pages/settings/CatalogSettings.jsx:7still lists'Lorcana'inCOMING_SOON_CATALOGS, so Catalog Settings renders Lorcana twice — once as a working toggle, once greyed out as "Coming Soon". The skill's registry sweep missed it because the sweep greps the game id (riftbound) and that file hardcodes the display label. This is the whole argument for derivation in one line.
1. Measurements
All commands were run from a checkout of 7330cd5. Every number below is mine, re-measured; where it disagrees with the prior session's estimate I say so.
1.1 The shape of PR #686
bash
git show --numstat --format="" 7330cd5 \
| awk '{printf "%-60s +%-6s -%s\n", $3, $1, $2}' | sort -t+ -k2 -rn36 files, 5,156 insertions. Subtracting the 525-line Phase 0 feasibility doc leaves 4,631 lines of code, which sort into four piles:
| Pile | Files | Lines | What |
|---|---|---|---|
| A Near-verbatim TCGCSV copies | 8 | 1,352 | 4 models (347), 2 importers (891), 2 workflows (114) |
| B Tests | 8 | 2,288 | plugin (678), catalog importer (428), catalog route (272), prices importer (223), setDetails route (185), ShopifyVariant round-trip (183), syncEnablement (167), priceCacheService (152) |
| C Registration + shared-path edits | 17 | 233 | see §1.5 |
| D Genuinely new game knowledge | 2 | 758 | plugins/lorcana/index.js (567), priceLookupService.js (191) |
Correction to the prior analysis: pile C is 17 files, not ~14, and it is not all "add my game to a list" — 150 of its 233 lines are the TCGCSV_GAMES refactor in catalogQueries.js / catalog.js / setDetails.js, which is one-time shared-path work already paid for. Pile D is 2 files, not ~5.
1.2 Model duplication — much higher than claimed
bash
for pair in "LorcanaSet RiftboundSet" "LorcanaProduct RiftboundProduct" \
"LorcanaPrice RiftboundPrice" \
"ShopifyLorcanaProductVariant ShopifyRiftboundProductVariant"; do
set -- $pair
diff <(sed 's/[Ll]orcana//g;s/LORCANA//g' server/models/$1.js \
| grep -vE '^\s*(\*|/\*\*|\*/|//)|^\s*$') \
<(sed 's/[Rr]iftbound//g;s/RIFTBOUND//g' server/models/$2.js \
| grep -vE '^\s*(\*|/\*\*|\*/|//)|^\s*$')
done| Model | Lines | Differing code lines | What differs |
|---|---|---|---|
LorcanaSet vs RiftboundSet | 43 | 0 | nothing but the category-id default and the name |
LorcanaProduct vs RiftboundProduct | 65 | 1 | inkType: String vs domain: String |
LorcanaPrice vs RiftboundPrice | 77 | 0 | nothing |
ShopifyLorcanaProductVariant vs ShopifyRiftboundProductVariant | 162 | 4 | vendor default; facet metafield name; two comments |
The prior estimate (81/81/70/90% identical) was measured with comments and on sorted lines. Stripping comments, the true figure is 5 differing code lines across 347, i.e. 98.6% identical.
A third witness for the Price model. PokemonPrice.js is also identical to LorcanaPrice.js modulo the game name and two trailing comments:
bash
diff <(sed 's/[Pp]okemon//g;s/POKEMON//g' server/models/PokemonPrice.js \
| grep -vE '^\s*(\*|/\*\*|\*/|//)|^\s*$') \
<(sed 's/[Ll]orcana//g;s/LORCANA//g' server/models/LorcanaPrice.js \
| grep -vE '^\s*(\*|/\*\*|\*/|//)|^\s*$')
# only difference: two inline comments on retail/buylistSo the price-model factory generalises from three games, not two. MTG's Price.js does not fit: it carries a second provider sub-schema (cardkingdom), plus sealedProductName / sealedProductCategory / setCode, and no game field. PokemonSet.js also does not fit the Set factory (it has id, cardCount, a non-unique groupId, and no modifiedOn).
1.3 Plugin duplication — the file diff understates it badly
A raw file diff says the two plugins are 61% identical, which reads like "different enough". That is an artifact of JSDoc and of Riftbound's four sealed-only methods. Comparing method by method, comment-stripped, with a line-level LCS (script: extract each method(...) { body, strip comments and game names, LCS per method):
Overlapping methods: A lines=290, LCS-identical=262 (90%)20 of the 32 overlapping methods are 100% identical: applyPricing, buildVariantProducts, findExistingProduct, findExistingProductLive, findRawPriceInTimeseries, generateHandle, generateSKU, getCardIdentityField, getProductModel, getSetCollectionSpec, getSourceCardId, getSourceCardIdField, nonFoilFinishes, normalizeCardNumber, preloadPrices, priceSource, dataSource, slugifySetName, transformToProduct, validateCard.
The methods that are not identical, and their entire variable surface:
| Method | Sim | What actually differs |
|---|---|---|
transformCardToProduct (61 lines) | 94% | 3 lines: vendor string; the tag array's facet fields (inkType vs domain.split(';')); one metafield key (ink_type vs domain) |
getMetafieldDefinitions | 94% | 1 line: the same facet metafield row |
findSet | 86% | the model name, and formatting |
selectVariantByFinish | 38% | Lorcana walks a foil preference list (3 finishes); Riftbound hardcodes normal/foil. Lorcana's implementation subsumes Riftbound's. |
rarities / finishes / displayName / gameCode | 58–77% | vocabulary — irreducible, §5.4 |
1.4 Importers and workflows
bash
diff <(sed 's/[Ll]orcana//g;s/LORCANA//g' server/scripts/data-loading/updateLorcanaPrices.js) \
<(sed 's/[Rr]iftbound//g;s/RIFTBOUND//g' server/scripts/data-loading/updateRiftboundPrices.js)Prices importer (418 / 398 lines). The entire semantic variable surface is the SUBTYPE_TO_FINISH map:
js
// Lorcana // Riftbound
{ 'Normal': 'normal', { Normal: 'normal',
'Cold Foil': 'coldFoil', Foil: 'foil' }
'Holofoil': 'holofoil' }…plus the category id (already plugin.dataSource.categoryId) and the model. Everything else — fetch, group iteration, per-day deleteMany-then-insertMany, bucket building, retention — is the same code.
Catalog importer (473 / 398 lines). Variable surface:
- category id (already a plugin constant)
- an extendedData key→field map:
{cardType: 'CardType', inkType: 'InkType'}vs{cardType: 'Card Type', domain: 'Domain'}— note the space, which is exactly the §5.4 trap and exactly what a manifest should force you to cite - an exclusion rule (
EXCLUDED_RARITY = 'None', Lorcana only) - the numeric-set-code collision warning (Lorcana only)
- the Phase-2 finish derivation — and here the two have already drifted:
js
// updateLorcanaCatalog.js — generic, driven off the plugin
const keys = Object.keys(Plugin.finishes).filter(f => typeof retail[f] === 'number');
// updateRiftboundCatalog.js — hardcoded, will not see a third finish
if (retail && typeof retail.normal === 'number') keys.push('normal');
if (retail && typeof retail.foil === 'number') keys.push('foil');That is a §5.1 One-Game Fix already present in main-to-be: add a finish to the Riftbound plugin and the importer silently ignores it. Same shape in updateRiftboundPrices.js. This is an argument for unifying sooner rather than later — the copies are already diverging, one of them defensively (Lorcana's for (const priceData of priceDocuments || []) vs Riftbound's unguarded of priceDocuments, which throws on undefined).
Workflows (54+60 / 42+50 lines). Differ in the workflow name, the cron minute, and the push.paths list. Nothing else.
priceLookupService.js (+191). The Lorcana block is a six-function family (getRaw…ForFinish, findRaw…InTimeseries, get…PriceForFinish, findValid…InTimeseries, getPricesFor…Card, update…VariantPrices). Comment-stripped it is 117 code lines vs Riftbound's 109, and diffing them name-erased shows five differing lines: the finish map, the default finishTypes, and two extra null guards Lorcana added. Fully parameterisable by (PriceModel, finishMap, cacheNamespace, defaultFinishes).
1.5 The hardcoded game lists
bash
grep -rn "lorcana\|Lorcana" server client --include=*.js --include=*.jsx \
| grep -v node_modules | grep -v "\.test\." \
| grep -vE "plugins/lorcana|models/(Shopify)?Lorcana|data-loading/updateLorcana"PR #686 touched 10 non-test list sites and 5 test-side ones. Phases 5–7 (sealed, buylist, marketplace) would add six more it did not need. Full table, with whether the registry could answer it:
| Site | Derivable from the registry? |
|---|---|
server/models/Store.js:87,90,93 — enabledCatalogs default, validator, error text | yes — getSupportedGames() |
server/schemas/shop.js:85 — z.enum | yes |
server/services/priceCacheService.js:395 — game allow-list | yes |
server/services/priceCacheService.js:436 — game→Price-model ternary | yes — via a new plugin.getPriceModel() |
server/schemas/catalog.js:141 — per-game facet param | yes — union of manifest facet fields |
server/routes/catalogQueries.js — TCGCSV_GAMES | yes, and it is the good pattern; derive its rows rather than delete the table |
client/src/context/StoreContext.jsx:21 — default enabledCatalogs | yes — from GET /api/games |
client/src/lib/rarityOptions.js — RARITY_OPTIONS | yes, verified 100% (§1.6) |
client/src/utils/gameLabels.js — GAME_LABELS | no, not as-is (§1.6) |
client/src/pages/settings/CatalogSettings.jsx:7 — COMING_SOON_CATALOGS | yes, and it is currently wrong |
client/src/pages/catalog/CatalogSealedPage.jsx:103,104 | yes — plugin.supportsSealed |
server/services/buylistQuoteService.js:383 — priceFns | yes — a plugin method |
server/models/Store.js:385, server/schemas/marketplaceConnections.js:28 (.max(3)) | yes — and .max(N) should be .max(getSupportedGames().length) |
server/marketplaces/cardtrader/{constants,mapping,index}.js | no — marketplace ids are the marketplace's, not the game's |
server/plugins/index.test.js:58 — expected gameCode list | no, deliberately — this is a forcing gate, keep it |
package.json — two data: scripts per game | yes — one --game CLI |
Evidence that hand-maintained lists rot: server/plugins/index.js exports MTGPlugin, PokemonPlugin, RiftboundPlugin at the bottom — but notLorcanaPlugin. Nothing failed.
1.6 Leverage point 5, verified — half right
bash
# derive [{value,label}] from plugin.rarities sorted by sortOrder, compare to RARITY_OPTIONS
node .tmpmeas/rar.cjs
mtg plugin= 6 client= 6 derivable-verbatim: true
pokemon plugin=25 client=25 derivable-verbatim: true
riftbound plugin= 6 client= 6 derivable-verbatim: true
lorcana plugin=10 client=10 derivable-verbatim: trueRARITY_OPTIONS is exactlyObject.entries(plugin.rarities).sort(by sortOrder).map(([value,{name}]) => ({value, label: name})) for all four games, all 47 rows. GET /api/games already ships rarities. Delete the file.
GAME_LABELS is not derivable:
| game | plugin.displayName | GAME_LABELS |
|---|---|---|
| mtg | Magic: The Gathering | MTG |
| pokemon | Pokemon TCG | Pokemon |
| riftbound | Riftbound TCG | Riftbound |
| lorcana | Disney Lorcana | Disney Lorcana |
These are a deliberate short label for narrow UI (sync history rows, catalog set cards, marketplace listings). Deleting gameLabels.js and substituting displayName would change "MTG" to "Magic: The Gathering" in five components. Fix: add shortName to the plugin and to getGamesInfo(), then derive. The prior analysis's claim that /api/games "already returns name" is true but insufficient.
Consumers already fetching /games: RadixShell.jsx:92, CatalogGameLayout, CatalogSettings, and four buylist pages — eight independent fetches, none shared. That is its own small problem (worth a GamesContext alongside PR A).
1.7 The tests — the prior analysis is wrong here
bash
# name-erased, comment-stripped, sorted-line diff, Lorcana vs Riftbound
index.test.js 542 vs 777 code lines identical 20%
updateLorcanaCatalog.test.js 347 vs 419 identical 30%
updateLorcanaPrices.test.js 183 vs 199 identical 34%
ShopifyLorcanaProductVariant.test.js 137 vs 85 identical 28%
grep -c "it(\|it\.each(" server/plugins/{lorcana,riftbound}/index.test.js
# lorcana: 86 riftbound: 85Same test count, same describe skeleton, 20–34% textual overlap. The tests are not copy-paste; they are re-derived per game because the assertions name real upstream values — 'Cold Foil', 'InkType', 'None', category 71, '2026-05-08', "529 of 3,301 cards have no normal printing".
Reading the 86 Lorcana plugin tests, roughly 50 assert shared behaviour (one variant per finish, deterministic finish sort, distinct barcodes from one sourceCardId, null on missing number, validateCard shape, applyPricing pre-sale flag, findExistingProduct fallback order, preloadPrices empty-map short-circuit) and roughly 36 assert this game's data. So the achievable saving on pile B is about 60%, and only by converting per-game test files into a shared contract suite plus a per-game fixture file — not by deleting anything.
2. Recommendation per leverage point
| # | Leverage point | Verdict | Confidence |
|---|---|---|---|
| 4 | Derive the hardcoded game lists from the registry | Do now | high — no new seam |
| 5 | Delete rarityOptions.js; derive gameLabels from a new shortName | Do now (amended) | high — verified |
| — | (new) Extract a TcgcsvGamePlugin base class from the 20 identical methods | Do now | high — extraction, not abstraction |
| 2 | Model factory | Do now for Price (3 witnesses); do at game 5 for Set/Product/ShopifyVariant (2) | high / medium |
| — | (new) Shared price-lookup function family in priceLookupService | Do now | high |
| 1 | One generic TCGCSV importer | Do now, as a pure refactor with no new game in the PR | medium-high |
| 3 | One matrix GitHub workflow | Don't do a matrix. Collapse to one looping workflow per phase, alongside #1 | high on "not a matrix" |
| 6 | Manifest-driven TcgcsvGamePlugin | Do at game 5 — discover the manifest from what #1/#2/the base class leave as arguments | deliberately deferred |
2.1 — Leverage 4: derive the game lists (do now)
This is not an abstraction. It replaces duplicated state with a read of the state that is already authoritative. There is no seam to get wrong, no generalisation from N=2, and it is individually revertible file by file.
It also fixes a shipped bug (COMING_SOON_CATALOGS) and closes the class of bug the add-new-game skill currently handles with a manual grep sweep. The durable win is turning "run the registry sweep" into "CI fails" — every list that becomes a derivation is a step the skill no longer has to instruct and a human no longer has to remember.
Not everything should be derived. Keep as hand-written gates:
server/plugins/index.test.js:58's expectedgameCodelist — deriving it makes the test tautological. Its job is to fail when a game registers.plugins/nonFoilFinishes.test.js'sCATALOG_FINISHES— see §5.3.- CardTrader's
GAME_IDS/SINGLES_CATEGORY_IDS/GAME_MAPPERS— those are the marketplace's vocabulary, live-verified per the skill's Phase 7. Not ours to derive.
But schemas/marketplaceConnections.js's .max(3) — the skill's named trap — should become .max(getSupportedGames().length), and its z.enum built from the registry. That removes the trap rather than documenting it.
2.2 — Leverage 5: rarity options and game labels (do now, amended)
Delete client/src/lib/rarityOptions.js; its three consumers (CardsFilterBar, SyncButton, CatalogSinglesPage) read rarities from the /api/games payload the shell already fetches. Verified byte-identical for all four games (§1.6).
Do not delete client/src/utils/gameLabels.js. Add shortName to BaseGamePlugin (defaulting to displayName), override it on the four plugins with today's GAME_LABELS values, ship it in getGamesInfo(), and reduce gameLabels.js to a lookup over the fetched payload. §5.4 applies to the plugin side: shortName is a UI choice, not upstream data, so it needs no citation — but it does need a test asserting the four current strings are unchanged, or this refactor silently renames "MTG" across the UI.
2.3 — New: extract TcgcsvGamePlugin (do now)
This is the change I would make first among the code ones, and it is not on the original list. Twenty methods are 100% identical between the two TCGCSV plugins (§1.3). Moving them to server/plugins/TcgcsvGamePlugin.js — a subclass of BaseGamePlugin, which the codebase already uses — is a mechanical extraction with the existing plugin test files as the oracle. No manifest, no factory, no new concept.
js
class LorcanaPlugin extends TcgcsvGamePlugin {
get gameId() { return 'lorcana'; }
get gameCode() { return 'LOR'; }
get categoryId() { return 71; }
get vendor() { return 'Ravensburger'; }
get rarities() { /* …cited vocabulary… */ }
get finishes() { /* …cited vocabulary… */ }
getProductModel() { return require('../../models/ShopifyLorcanaProductVariant'); }
get facets() { /* …two rows… */ }
}Riftbound and Lorcana each drop ~115 lines. Crucially, this is where the manifest is discovered: whatever ends up as a getter on the subclass is, by definition, the manifest's field list — established by extraction rather than by prediction. §5.9's warning is about designing config forward; reading it backwards off a diff is the opposite activity.
Adopt Lorcana's selectVariantByFinish (preference-list) as the base implementation — it subsumes Riftbound's two-finish special case, and a three-finish game already exists.
2.4 — Leverage 2: model factory
createTcgcsvPriceModel(gameId) — do now. Three witnesses, zero differing code lines across all three. The signature is (gameId) and nothing else; the collection name, game default, time-series options, retention, and the {game, slug, timestamp} index are all mechanical. Migrate PokemonPrice, RiftboundPrice, LorcanaPrice. Leave Price.js (MTG) alone — it has a second provider and three extra fields.
Set / Product / ShopifyVariant factories — do at game 5. Two witnesses each, and PokemonSet actively does not fit (different key fields). The variable surface is small and known (category id; the facet field list; the vendor default), but "small and known across two samples" is precisely the §5.9 condition. Build game 5's models by hand from the manifest fields the base-class extraction surfaced, then factor all three together — three samples, and the third one was not designed to match the first two.
2.5 — New: shared price-lookup family (do now)
priceLookupService.js grows a six-function block per game (191 lines for Lorcana) whose variable surface measured at five lines. Replace with one factory returning the six functions, parameterised by (PriceModel, finishMap, cacheNamespace, defaultFinishes, gameId). Riftbound and Lorcana adopt it; Pokemon's and MTG's blocks stay as they are for now (unmeasured — do not fold them in on assumption).
Take Lorcana's null guards as the shared behaviour: Riftbound's for (const priceData of priceDocuments) throws on undefined, a latent bug the copy-paste introduced.
This also kills a Phase-6 trap. buylistQuoteService.js's hardcoded priceFns map exists only because these functions are per-game module exports. If the family is constructed per plugin, plugin.getPriceFns() replaces the map and the skill's "it throws on the first quote" red flag stops being possible.
2.6 — Leverage 1: one generic TCGCSV importer (do now, as a pure refactor)
The variable surface is fully enumerated in §1.4 and it is small. More importantly, the two copies have already drifted, and the drift is the repo's most-repeated bug class: Riftbound's importer cannot see a third finish because it hardcodes normal/foil in two places, while Lorcana's derives from plugin.finishes. Waiting for game 5 means shipping game 5 next to two divergent copies and choosing which to clone.
Shape: server/scripts/data-loading/tcgcsv/{catalog,prices}.js exporting runCatalog(plugin, deps) / runPrices(plugin, deps), plus a thin updateTcgcsvCatalog.js --game <id> CLI. Keep the existing _setDeps DI seam — vi.mock of server source is inert under this Vitest config (CLAUDE.md §3), so DI is the only workable test seam.
The condition on doing this now: the PR adds no new game and does not touch updateRiftboundCatalog.test.js, updateRiftboundPrices.test.js, updateLorcanaCatalog.test.js or updateLorcanaPrices.test.js. Those 1,370 lines of tests are the oracle. If the refactor needs them edited, the seam is wrong and the PR should stop. Two exceptions are expected and must be named individually in the PR description: the Riftbound finish-derivation tests move to the plugin-driven behaviour, and the for…of null guard changes.
Keep the numeric-set-code collision warning as an unconditional check in the shared catalog importer — it costs nothing on games whose codes aren't numeric, and it is the kind of check that only gets written once.
2.7 — Leverage 3: workflows — collapse, but not into a matrix
A strategy.matrix over 100 games means 100 runner VMs each doing actions/checkout + npm ci, 100 simultaneous Mongo Atlas connections, and 100 simultaneous TCGCSV fetches. It also loses the per-game push.paths filter that today makes update-lorcana-catalog.yml re-run when only Lorcana's importer changes, and it loses the deliberate cron staggering (Lorcana catalog 6:30, prices 6:45 — chosen because 5:45 and 6:15 were taken by Riftbound).
Recommendation: one update-tcgcsv-catalogs.yml and one update-tcgcsv-prices.yml, each a single job that checks out once, installs once, and loops the registry-driven game list sequentially in one process — the --game CLI from §2.6, or an --all mode. Per-game try/catch so one game's upstream shape change cannot abort the other 99, with a non-zero exit only after every game has been attempted. Path filter becomes server/scripts/data-loading/tcgcsv/** + server/plugins/**.
Revisit when the sequential loop approaches the current timeout-minutes: 30. At that point chunk the games and matrix over chunks with max-parallel, not over games.
Do this alongside §2.6, not before — the shared CLI is the prerequisite.
2.8 — Leverage 6: the manifest (do at game 5)
Do not write a manifest schema now. Do §2.3–§2.6 first; the manifest is then just "the set of getters the base class asks its subclasses for", already validated by two working games. Build game 5 by writing only that manifest and see what it can't express. Then, and only then, formalise it.
§3 writes the manifest out for both games as it would look after §2.3–§2.6, so the shape can be judged now — but as a prediction to be tested at game 5, not as something to build against today.
3. The proposed manifest, written out for both games
Predicted shape after the extraction steps. Every value below is copied from 7330cd5, not invented. The cite fields are the §5.4 forcing mechanism made structural: a manifest field that must match upstream data carries the query and date that produced it, in the file, next to the value.
3.1 Lorcana
js
// server/plugins/lorcana/manifest.js
module.exports = {
gameId: 'lorcana',
gameCode: 'LOR', // unique — assertUniqueGameCodes() throws on collision
displayName: 'Disney Lorcana',
shortName: 'Disney Lorcana', // §2.2 — UI short label, not upstream data
vendor: 'Ravensburger', // Shopify product vendor
source: {
kind: 'tcgcsv-json', // /products + /prices endpoints, 1 row per product
categoryId: 71,
cite: 'https://tcgcsv.com/tcgplayer/categories, 2026-08-31',
},
// §5.3 slug invariant: catalog identity === price slug.
identity: {
catalogField: 'tcgplayerProductId',
priceSlug: 'String(tcgplayerProductId)',
cite: 'sweep of all 20 category-71 groups 2026-08-31: 0 price rows referencing a missing product',
},
setCodes: {
from: 'abbreviation', // TCGCSV group abbreviation, used verbatim as set_code
case: 'upper',
numeric: true, // '1'-'14' — enables the groupId-collision warning
// and forces ordered findSet queries, not one $or
cite: 'TCGCSV /tcgplayer/71/groups + Lorcast api.lorcast.com/v0/sets, 2026-08-31: all 20 unique and non-empty',
},
// Ordered by the scarcity ladder. Keys are the ids buylist rarityRates uses.
rarities: {
common: { name: 'Common', sortOrder: 1 },
uncommon: { name: 'Uncommon', sortOrder: 2 },
rare: { name: 'Rare', sortOrder: 3 },
'super rare': { name: 'Super Rare', sortOrder: 4 },
legendary: { name: 'Legendary', sortOrder: 5 },
epic: { name: 'Epic', sortOrder: 6 },
iconic: { name: 'Iconic', sortOrder: 7 },
enchanted: { name: 'Enchanted', sortOrder: 8 },
quest: { name: 'Quest', sortOrder: 9 },
promo: { name: 'Promo', sortOrder: 10 },
},
raritiesCite: 'distinct Rarity over all 20 groups 2026-08-31: Common 957, Uncommon 714, '
+ 'Rare 635, Super Rare 240, Enchanted 222, Promo 203, Legendary 161, Epic 90, '
+ 'Quest 77, Iconic 10',
// Rows carrying this Rarity are real TCGCSV values but not tradeable singles.
excludeRarities: ['None'],
excludeRaritiesCite: '165 puzzle inserts across all groups; populated extendedData, '
+ 'Rarity === "None", no Number (10/10 on group 24617, 2026-08-31)',
// id -> { display name, TCGCSV subTypeName, foil?, sortOrder }
// Drives: plugin.finishes, nonFoilFinishes, SUBTYPE_TO_FINISH in the prices
// importer, FINISH_SORT_ORDER in the transform, the price-lookup finish map,
// and selectVariantByFinish's preference order.
finishes: {
normal: { name: 'Normal', subTypeName: 'Normal', foil: false, sortOrder: 1 },
coldFoil: { name: 'Cold Foil', subTypeName: 'Cold Foil', foil: true, sortOrder: 2 },
holofoil: { name: 'Holofoil', subTypeName: 'Holofoil', foil: true, sortOrder: 3 },
},
finishesCite: 'distinct subTypeName over all 20 groups 2026-08-31: '
+ 'Normal 3083, Cold Foil 2723, Holofoil 452',
// Extra columns flattened off extendedData. One row generates: the raw model
// field, the importer's flatten line, the Shopify-ready metafield, the
// metafield definition, the tag contribution, and TCGCSV_GAMES.facetField.
facets: [
{ field: 'cardType', extendedDataKey: 'CardType',
metafield: { key: 'card_type', name: 'Card Type' }, tag: true },
{ field: 'inkType', extendedDataKey: 'InkType',
metafield: { key: 'ink_type', name: 'Ink Type' }, tag: true, browseFacet: true },
],
facetsCite: "TCGCSV extendedData keys for category 71 have NO space ('CardType', 'InkType') "
+ "while sibling keys do ('Cost Ink', 'Lore Value') — read from a real response "
+ "2026-08-31, not guessed from Riftbound's 'Card Type'",
capabilities: { sealed: false, sealedCategoryTaxonomy: false },
};3.2 Riftbound — the same shape, no new fields
js
// server/plugins/riftbound/manifest.js
module.exports = {
gameId: 'riftbound',
gameCode: 'RFT',
displayName: 'Riftbound TCG',
shortName: 'Riftbound',
vendor: 'Riot Games',
source: { kind: 'tcgcsv-json', categoryId: 89, cite: '…' },
identity: { catalogField: 'tcgplayerProductId',
priceSlug: 'String(tcgplayerProductId)', cite: '…' },
setCodes: { from: 'abbreviation', case: 'upper', numeric: false, cite: '…' },
rarities: {
common: { name: 'Common', sortOrder: 1 },
uncommon: { name: 'Uncommon', sortOrder: 2 },
rare: { name: 'Rare', sortOrder: 3 },
epic: { name: 'Epic', sortOrder: 4 },
showcase: { name: 'Showcase', sortOrder: 5 },
promo: { name: 'Promo', sortOrder: 6 },
},
raritiesCite: '…',
excludeRarities: [], // Riftbound has no puzzle-insert analogue
finishes: {
normal: { name: 'Normal', subTypeName: 'Normal', foil: false, sortOrder: 1 },
foil: { name: 'Foil', subTypeName: 'Foil', foil: true, sortOrder: 2 },
},
finishesCite: '…',
facets: [
{ field: 'cardType', extendedDataKey: 'Card Type', // note the space
metafield: { key: 'card_type', name: 'Card Type' }, tag: true },
{ field: 'domain', extendedDataKey: 'Domain',
metafield: { key: 'domain', name: 'Domain' },
tag: true, browseFacet: true, multiValueSeparator: ';' }, // "Mind;Chaos"
],
capabilities: { sealed: true, sealedCategoryTaxonomy: false },
};3.3 Coverage check against the measured diff
Every difference found in §1.2–§1.4 is expressed:
| Measured difference | Manifest field |
|---|---|
LorcanaProduct.inkType vs RiftboundProduct.domain | facets[].field |
ShopifyVariant vendor default | vendor |
ShopifyVariant facet metafield | facets[].metafield |
| category-id defaults (71 / 89) | source.categoryId |
| importer extendedData key map, incl. the space | facets[].extendedDataKey + facetsCite |
importer SUBTYPE_TO_FINISH | finishes[].subTypeName |
| importer exclusion rule | excludeRarities |
| importer numeric-collision warning | setCodes.numeric |
transformCardToProduct tag composition, incl. the ; split | facets[].tag + multiValueSeparator |
getMetafieldDefinitions facet row | facets[].metafield |
selectVariantByFinish preference order | finishes[].foil + sortOrder |
rarities, finishes, displayName, gameCode | direct |
| price-lookup finish map + default finishes | derived from finishes |
| workflow name / paths | derived from gameId |
Nothing in the measured diff is unexpressed. That is the evidence the shape covers both — and the reason to test it at game 5 rather than build it now: covering two known samples is what a manifest derived from those two samples will always do.
3.4 What the manifest deliberately does not cover
sealed (Phase 5), buylist (Phase 6) and marketplace (Phase 7). Riftbound's four sealed methods are 126 lines with one witness. Leave them hand-written on the subclass until a second TCGCSV game ships sealed.
4. What stays bespoke, and why
MTG — entirely. Different source kind (dataSource.type: 'file', MTGJSON AllPrintings.json, not an HTTP catalog per group). SetModel embeds the whole MTGJSON set as data with a cards array. Price.js carries two provider sub-schemas (tcgplayer + cardkingdom), plus sealedProductName, sealedProductCategory and setCode, and no game field. It also owns the treatment table, the sealed category taxonomy (hasSealedCategoryTaxonomy is MTG-only), the LINKED_ENTRY_LAYOUTS physical-card-count logic, and the TCGplayer inventory import. Forcing MTG into a TCGCSV manifest would mean adding option flags that exactly one game sets — the §5.9 pattern.
Pokemon — mostly. It is TCGCSV (category 3), but through ProductsAndPrices.csv per group, one row per (card, finish), not the JSON /products + /prices pair. It needs buildCardIdentityMap because one collector number can cover several TCGCSV products (Poke Ball / Master Ball patterns, merged promo groups), which is why its identity key is the composite setID-number and not productId (§5.3, PR #260). Its sets are keyed tcgcsv-{groupId}; PokemonSet has id, cardCount, and a non-unique groupId. Its plugin implements transformRowToProduct, not transformCardToProduct.
But its Price model does fit — measured byte-identical to LorcanaPrice — so Pokemon should adopt createTcgcsvPriceModel even though nothing else about it generalises. Partial adoption is the point: the factories are per-artifact, not per-game.
Per-game, always, for every game including manifest ones:
- The Phase 0 feasibility doc (525 lines for Lorcana). Non-negotiable — it is where the identity-key decision and the §5.3 invariant get written down.
- The vocabularies: rarities, finishes,
subTypeNamemapping, facet fields, exclusion rules, set-code scheme, vendor. This is the manifest, and it is irreducible §5.4 content. - The fixtures: real upstream rows the shared tests run against (§5.3).
- Genuine overrides. Lorcana's ordered
findSet(abbreviation before numeric groupId, two queries not one$or) is currently a Lorcana behaviour; ifsetCodes.numericdrives it in the base class, fine — but a game whose title format or handle scheme genuinely differs overrides the method, and the base class must let it.
5. Testing strategy — earning the inverted blast radius
Today a bad importer breaks one game. After this, it breaks all of them. Six things have to be true before that trade is acceptable.
5.1 The existing per-game tests are the refactor's oracle
Every extraction PR (§2.3–§2.6) must leave updateRiftbound*.test.js, updateLorcana*.test.js, plugins/{riftbound,lorcana}/index.test.js and the two Shopify*ProductVariant.test.js files unedited and green. 1,370+ importer-test lines and 1,614 plugin-test lines already encode the behaviour; if the shared code needs them changed, the seam is wrong. Deliberate exceptions get named individually in the PR description with the reason.
This is the entry price and it is cheap, because those tests already exist.
5.2 A contract suite that runs per registered game
The repo already has the pattern — six registry-driven tests (pricingDefaults, metafieldDefinitions, priceTimeseries, nonFoilFinishes, index, sealedQuickAddService.collections) that fail the moment a plugin registers incomplete. Extend it: the ~50 shared-behaviour tests identified in §1.7 move into server/plugins/tcgcsv/contract.test.js, iterating getSupportedGames().filter(g => getPlugin(g).source?.kind === 'tcgcsv-json').
Same for the shared importer: one tcgcsv/catalog.contract.test.js that runs fetch → flatten → classify → exclude → transform → Phase-2 grouping against each game's recorded fixture.
5.3 The forcing property must move, not die
plugins/nonFoilFinishes.test.js works because of one line:
js
expect(new Set(getSupportedGames())).toEqual(new Set(Object.keys(CATALOG_FINISHES)));A new game cannot pass CI without a hand-written CATALOG_FINISHES row citing a real distinct() query. Preserve this exactly, relocated. Proposal:
- Each manifest-driven plugin ships
server/plugins/<game>/fixtures.js: raw recorded upstream rows (one/groupsentry, three/productsentries covering the finish and rarity edge cases, one/pricesbucket per finish), plus the declaredcatalogFinishes: { nonFoil, foil }table, each with the query and date that produced it. - The shared suite discovers fixture files by directory scan and asserts
new Set(tcgcsvGames()) === new Set(discoveredFixtures). A game with no fixtures file fails CI, exactly as today.
The failure mode to design against is tautology. If the fixture stores derived values, the shared test degenerates into "manifest equals manifest" and asserts nothing. The fixture must hold the raw upstream JSON, and the assertion must be "manifest applied to this raw row produces this expected catalog doc / Shopify-ready doc". That is what makes a wrong extendedDataKey: 'InkType' vs 'Ink Type' fail — the exact §5.4 bug the current per-game tests catch.
Any PR in this programme that makes a registry-driven test derive its expected values from the thing it is checking should be rejected on that basis alone.
5.4 Golden files per game
Commit, per game, the expected output of: (a) transformProduct over the fixture rows, (b) transformCardToProduct over the same, (c) the Phase-2 variant grouping. A change to shared code that alters any game's output then shows up as N diffs in review rather than as zero. This is the specific control for the inverted blast radius: it makes "I changed the shared importer and nothing failed" impossible when something did change.
At 100 games these are small (a few KB each) and their diff is the review artifact — if a shared change is meant to affect one game, exactly one golden file should move.
5.5 Operational isolation
The shared daily runner must fail per game, not per run: try/catch around each game, accumulate failures, exit non-zero only after every game has been attempted, and log the failing game id prominently. Today that isolation is free (one workflow per game). A shared runner has to buy it back explicitly, and it is the difference between "Lorcana's upstream changed shape" and "no game got prices today".
Carry the §5.6 constraints forward unchanged into the shared code: no server-side $sort on any of the time-series price collections, no await Model.find({slug: {$in: […]}}) materialisation, per-slug caps via loadLatestSnapshotsBySlug. A shared importer that gets this wrong gets it wrong for 100 games at once on a 2 GB worker.
5.6 Coverage
Shared modules carry N games' load-bearing behaviour, so 70% is not enough for them. Recommend per-file thresholds above the ratchet for server/plugins/TcgcsvGamePlugin.js, server/plugins/tcgcsv/** and server/scripts/data-loading/tcgcsv/** — 90% statements/lines.
(Aside, unrelated to this design: the thresholds block in vitest.config.js is now correctly nested and is operative. CLAUDE.md §7's note that the gate is silently dead is stale and should be corrected separately.)
6. Migration path
Seven PRs, each independently shippable and revertible. No step rewrites MTG or Pokemon, and no step requires the other three games to move at the same time.
| PR | Scope | Touches game logic? | Risk |
|---|---|---|---|
| A | Derive game lists (§2.1) + shortName + delete rarityOptions.js (§2.2). Fixes the COMING_SOON_CATALOGS bug. | no | low |
| B | TcgcsvGamePlugin base class; Riftbound + Lorcana extend it (§2.3). Plugin test files unedited. | yes, 2 games | low-med |
| C | createTcgcsvPriceModel; Pokemon + Riftbound + Lorcana adopt (§2.4). | no behaviour | low |
| D | Shared price-lookup family; Riftbound + Lorcana adopt; priceFns becomes plugin.getPriceFns() (§2.5). | yes, 2 games | med |
| E | Shared TCGCSV importers + --game CLI (§2.6). Importer test files unedited. | yes, 2 games | med-high |
| F | Collapse the four TCGCSV workflows into two looping ones (§2.7). Requires E. | no | low |
| G | At game 5: build it manifest-only on what B–E left as arguments. Port Riftbound/Lorcana to manifests in the same PR only if game 5 needed no new escape hatch; otherwise leave them and port at game 6. | yes | the real test |
A–C are safe to run in parallel; D–F are sequential. If the programme is abandoned after any PR, the codebase is in a coherent state — that is the property that makes it worth doing incrementally rather than as one refactor.
The gate on G: if game 5 needs an override the manifest can't express, that is the manifest telling you it was derived from too few samples. Add the override as a subclass method, ship game 5, and formalise at game 6. Do not widen the manifest to fit one game — that is how config fields become §5.9 dead scaffold.
The add-new-game skill needs a rewrite after A–F: the registry-sweep table shrinks to the genuinely-hand-written gates, and Phase 1's model + importer checklist collapses to "write the manifest and the fixtures". The phase ordering trap (registration lands in Phase 3, browse depends on it) survives unchanged and must be restated, because registration is still what makes the registry-driven contract suite run.
7. What this would have saved on PR #686
Measured against 7330cd5: 36 files / 4,631 code lines (excluding the 525-line Phase 0 doc, which stays).
Full end state (through PR G)
| Artifact | #686 | End state | Saved | Basis |
|---|---|---|---|---|
| 4 models | 347 | ~30 | 317 | 5 differing code lines measured (§1.2) |
| 2 importers | 891 | 0 | 891 | variable surface = field map + finish map + exclusion + category id (§1.4) |
| 2 importer test files | 651 | ~60 | 591 | logic shared; fixtures stay |
| 2 workflows | 114 | 0 | 114 | differ only in cron + name |
priceLookupService block | 191 | 0 | 191 | 5 differing code lines of 109 (§1.4) |
plugins/lorcana/index.js | 567 | ~180 | 387 | 20/32 methods 100% identical; 90% LCS (§1.3) |
plugins/lorcana/index.test.js | 678 | ~200 | 478 | ~50 of 86 tests assert shared behaviour (§1.7) |
ShopifyLorcanaProductVariant + test | 345 | ~50 | 295 | 4 differing lines |
| 3 route/service test files | 609 | ~180 | 429 | shared path exists; suites parameterise |
syncEnablement.test.js | 167 | ~40 | 127 | registry-driven contract |
| 17 registration/list files | 233 | ~20 | 213 | §2.1, §2.2 |
| Total | 4,631 | ~760 | ~3,870 | ≈ 84% |
36 files → ~5: manifest.js, fixtures.js, index.js (overrides, possibly near-empty), the golden-file snapshot, and the Phase 0 design doc. No package.json row, no workflows, no models, no importers, no route edits.
Of the ~760 residual lines, roughly 350 are fixtures, 90 the manifest, 150 genuine overrides, and the rest golden files — i.e. the residue is almost entirely §5.4 content, which is the correct floor.
Near-term only (PRs A–D, no manifest, no shared importer)
| Item | Saved |
|---|---|
TcgcsvGamePlugin base class off the plugin | ~115 |
priceLookupService family | 191 |
createTcgcsvPriceModel (LorcanaPrice) | 77 |
LorcanaSet via the base's set handling | 43 |
rarityOptions.js row | 16 |
| derivable registration lists | ~40 |
| Total | ~480 |
About 10% of the PR, for four low-risk PRs that touch no MTG or Pokemon code. The large number is unavailable until the importer and the manifest land, and the manifest should not land until game 5 exists to test it.
The honest caveat
The ~3,870 figure assumes games 5, 6 and 7 are TCGCSV-JSON-shaped like Riftbound and Lorcana. If the next few games are CSV-shaped like Pokemon, or file-shaped like MTG, the manifest covers none of them and the saving is the near-term ~480 plus whatever a second shared shape is worth. Before committing to PR G, run the Phase 0 feasibility step for the next two or three intended games and check dataSource.type. That is a cheap check and it determines whether this programme is worth 84% or 10%.
8. Open questions
- What are games 5–8? Their
dataSource.typedecides whether the manifest is the right end state (§7 caveat). If two of them are CSV-shaped, the design should grow a secondsource.kindbefore PR G, not after. - Is the
COMING_SOON_CATALOGSdouble-render worth a fast-follow on #686, or fold it into PR A? - PR E's blast radius. Is it acceptable to share the importer for the two smallest games' daily feeds now, with golden files and per-game isolation as the controls — or should that wait until 5+ games are paying for it?
- Skill update cadence: fold the
add-new-gamerewrite into each PR, or one skill-update PR at the end of A–F?