Appearance
TCGplayer Import — PR 1 (Parse, Match, Preview) Implementation Plan
For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (
- [ ]) syntax for tracking.
Goal: A merchant uploads their TCGplayer Pricing Custom Export and gets back a persisted, honest report — what we matched, what we didn't and why, how many products this would create, whether their plan covers it, and how our pricing compares to theirs — with zero writes to Shopify.
Architecture: Three new server modules with one responsibility each. server/utils/tcgplayerCsv.js owns the file format (header detection, row parse, bucket classification) and absorbs the TCGplayer condition helpers currently living inside buylistQuoteService. server/services/tcgcsvGroupService.js owns the Set Name → groupId → set code bridge and its Redis cache. server/services/tcgImportService.js orchestrates: parse → classify → resolve sets → match against the catalog → persist → summarize. Two new Mongo collections hold the import and its rows. One route surface, four endpoints, all Zod-validated.
Tech Stack: Node.js/Express (CommonJS), Mongoose, Zod, csv-parser (already a dependency), ioredis via config/redis, React 18 + retroui + Tailwind, Vitest (test files are ESM).
Design spec: docs/superpowers/specs/2026-08-04-tcgplayer-inventory-import-design.md — read its "Source data — verified facts" section before starting. Every literal in this plan traces to a real file or a live API response documented there.
Scope: This plan is PR 1 of four. PRs 2 (execute singles), 3 (price lock), and 4 (sealed) get their own plans, matching how 2026-07-30-sealed-upc-pr1.md / 2026-07-31-sealed-upc-pr2-repair.md were split. Nothing in this plan writes to Shopify.
Global Constraints
- Never invent a literal (§5.4). Every column name, condition value, and set name below was read from a real export:
TCGplayer__Pricing_Custom_Export_20260804_020845.csv, 8,514 rows, captured 2026-08-04. The TCGCSV group list was read live fromhttps://tcgcsv.com/tcgplayer/1/groups(453 MTG groups) on the same date. TCGplayer Idis never a join key. Measured: 1,403 of 1,403 multi-condition groups carry a different id per condition. It is SKU-level. Store it inrawfor diagnostics; join on nothing.- CSV
Rarityis never mapped to our rarity. The file writes TCGplayer'sR/M/U/C/P/S/L/T;metafields.rarityis"Mythic"/"Rare". Rarity always comes from the matched catalog document. gameis never defaulted (§5.5). No|| 'mtg',?? 'mtg', or= 'mtg'anywhere in this diff.gameflows URL param → client → schema → service.Unopenedis intercepted before condition normalization.normalizeCondition('Unopened')returns'nm'. A sealed box classified as a Near Mint single is the §5.4 failure shape; Task 1 has a test that fails if the interception is removed.- The name fallback is scoped to the resolved set. Never a global
findOneon card name. An ambiguous name resolves tounmatched, never an arbitrary pick. - No user-controlled outbound fetch. TCGCSV URLs are built from a hardcoded host plus a
groupIdwe resolved ourselves. CSV text never reaches a URL. - TCGCSV requires an identifying User-Agent.
lgs-forge/1.0 (ufkesba@gmail.com)— verified: an unidentified agent gets a plaintext block message, not JSON, which then failsJSON.parsewith a confusingUnexpected token 'A'. - No new per-game plugin, per-game model, or importer is touched — §5.1 parity is satisfied by declaration. State in the PR: mtg = implemented; pokemon = exempt (no real export,
Product Lineliteral unverified); riftbound = exempt (same). - Every new persisted field is declared field-by-field with a round-trip test (§5.2). Both new models ship with one.
- Server code is CommonJS (
require); test files are ESM (import). - DI is
_setDeps/_resetDeps/getDeps(), asbuylistQuoteService.jsdoes.vi.mockof a server module is silently inert under this repo's Vitest config — do not reach for it. - Coverage ≥70% on all four metrics for every modified file, read from the per-file table by hand. Note:
vitest.config.jsnow carries a real nestedthresholdsblock (lines 51 / functions 59 / branches 44 / statements 51, a repo-wide ratchet added 2026-07-27), sonpm run test:coverageCAN fail — but that gate is global, not per-file, so it does not enforce the 70% per-file bar. CLAUDE.md §7's claim that the gate is silently inoperative is stale; it describes the pre-2026-07-27 flat-key config. Never lower a threshold to make a run pass. npm run lintclean, zero new warnings.npm run buildmust pass (client is touched).- Commit messages: one imperative sentence, merchant-visible outcome, sentence case, no trailing period.
Task 1: The TCGplayer CSV format module
Extract the three TCGplayer condition helpers out of buylistQuoteService into a shared module, and add the Pricing-Custom-Export header detection, row parse, and bucket classification.
Files:
- Create:
server/utils/tcgplayerCsv.js - Create:
server/utils/tcgplayerCsv.test.js - Modify:
server/services/buylistQuoteService.js(delete the three helpers, import them instead)
Interfaces:
Produces:
stripFoilSuffix(condition: string): stringtcgplayerFinish(condition: string): 'foil'|'nonfoil'|nullnormalizeCondition(raw: string): 'nm'|'lp'|'mp'|'hp'|'damaged'assertPricingExportHeader(headerLine: string): void— throwsTcgImportErrorwith.codeon a non-pricing-export headerclassifyRow(row: object): { status, reason, parsed }wherestatusis'zero_qty'|'unsupported_line'|'sealed'|'single'andparsedis{ finish, condition, quantity }TcgImportError—.name === 'TcgImportError',.code: stringMAX_IMPORT_ROWS = 100000- Used by Tasks 2, 4, and 5.
[ ] Step 1: Write the failing tests
Create server/utils/tcgplayerCsv.test.js:
javascript
import { describe, it, expect } from 'vitest';
import {
stripFoilSuffix,
tcgplayerFinish,
normalizeCondition,
assertPricingExportHeader,
classifyRow,
TcgImportError,
MAX_IMPORT_ROWS
} from './tcgplayerCsv.js';
// The exact header line of the reference export, captured 2026-08-04.
const REAL_HEADER = 'TCGplayer Id,Product Line,Set Name,Product Name,Title,Number,Rarity,Condition,TCG Market Price,TCG Direct Low,TCG Low Price With Shipping,TCG Low Price,Total Quantity,Add to Quantity,TCG Marketplace Price,My Store Reserve Quantity,My Store Price,Photo URL';
// The TCGplayer *collection* export — shares "TCGplayer Id" with the pricing
// export, which is why detection cannot key on that column alone.
const COLLECTION_HEADER = 'TCGplayer Id,Product Line,Set Name,Product Name,Title,Number,Rarity,Condition,Quantity,Added';
function row(overrides = {}) {
return {
'TCGplayer Id': '376073',
'Product Line': 'Magic',
'Set Name': '10th Edition',
'Product Name': 'Adarkar Wastes',
Title: '',
Number: '347',
Rarity: 'R',
Condition: 'Lightly Played',
'TCG Market Price': '4.63',
'Total Quantity': '1',
'TCG Marketplace Price': '5.3700',
...overrides
};
}
describe('condition and finish parsing', () => {
// All 10 condition values that actually occur in the reference export.
it.each([
['Near Mint', 'nm', 'nonfoil'],
['Near Mint Foil', 'nm', 'foil'],
['Lightly Played', 'lp', 'nonfoil'],
['Lightly Played Foil', 'lp', 'foil'],
['Moderately Played', 'mp', 'nonfoil'],
['Moderately Played Foil', 'mp', 'foil'],
['Heavily Played', 'hp', 'nonfoil'],
['Heavily Played Foil', 'hp', 'foil'],
['Damaged', 'damaged', 'nonfoil'],
['Damaged Foil', 'damaged', 'foil']
])('parses %s as condition %s / finish %s', (input, condition, finish) => {
expect(normalizeCondition(stripFoilSuffix(input))).toBe(condition);
expect(tcgplayerFinish(input)).toBe(finish);
});
it('returns null finish for a blank condition, rather than guessing nonfoil', () => {
expect(tcgplayerFinish('')).toBeNull();
});
it('falls back to nm for an unrecognized condition instead of throwing', () => {
expect(normalizeCondition('Slightly Chewed')).toBe('nm');
});
it('does not resolve a prototype key through the alias map', () => {
expect(normalizeCondition('__proto__')).toBe('nm');
expect(normalizeCondition('constructor')).toBe('nm');
});
});
describe('assertPricingExportHeader', () => {
it('accepts the real pricing-export header', () => {
expect(() => assertPricingExportHeader(REAL_HEADER)).not.toThrow();
});
it('rejects the collection export, which also has a TCGplayer Id column', () => {
try {
assertPricingExportHeader(COLLECTION_HEADER);
throw new Error('should have thrown');
} catch (e) {
expect(e).toBeInstanceOf(TcgImportError);
expect(e.code).toBe('WRONG_TCGPLAYER_EXPORT');
expect(e.message).toMatch(/Pricing Custom Export/i);
}
});
it('rejects an unrelated CSV', () => {
try {
assertPricingExportHeader('name,qty,price');
throw new Error('should have thrown');
} catch (e) {
expect(e.code).toBe('WRONG_TCGPLAYER_EXPORT');
}
});
it('is case- and space-insensitive about column names', () => {
expect(() => assertPricingExportHeader(REAL_HEADER.toLowerCase())).not.toThrow();
expect(() => assertPricingExportHeader(REAL_HEADER.replace(/,/g, ', '))).not.toThrow();
});
});
describe('classifyRow', () => {
it('buckets a zero-quantity row before anything else', () => {
// 5,570 of the reference file's 8,514 rows land here.
const result = classifyRow(row({ 'Total Quantity': '0' }));
expect(result.status).toBe('zero_qty');
});
it('treats a blank quantity as zero rather than one', () => {
expect(classifyRow(row({ 'Total Quantity': '' })).status).toBe('zero_qty');
});
it('buckets a non-Magic product line as unsupported, naming the line', () => {
const result = classifyRow(row({ 'Product Line': 'Pokemon' }));
expect(result.status).toBe('unsupported_line');
expect(result.reason).toMatch(/Pokemon/);
});
it('buckets Unopened as sealed BEFORE condition normalization', () => {
// Guard test: normalizeCondition('Unopened') returns 'nm', so without
// this interception a sealed booster box imports as a Near Mint single.
const result = classifyRow(row({
Condition: 'Unopened',
'Product Name': "Ugin's Fate - Event Booster Pack",
Number: '',
Rarity: '',
'Total Quantity': '2'
}));
expect(result.status).toBe('sealed');
expect(result.parsed.condition).not.toBe('nm');
});
it('classifies an ordinary stocked row as a single with parsed finish and condition', () => {
const result = classifyRow(row({ Condition: 'Lightly Played Foil', 'Total Quantity': '3' }));
expect(result.status).toBe('single');
expect(result.parsed).toEqual({ finish: 'foil', condition: 'lp', quantity: 3 });
});
it('reads columns case-insensitively', () => {
const lower = { 'product line': 'Magic', 'total quantity': '1', condition: 'Near Mint' };
expect(classifyRow(lower).status).toBe('single');
});
});
describe('MAX_IMPORT_ROWS', () => {
it('is far above the reference export and is not the buylist paste guard', () => {
expect(MAX_IMPORT_ROWS).toBe(100000);
expect(MAX_IMPORT_ROWS).not.toBe(500);
});
});- [ ] Step 2: Run the tests to verify they fail
Run: npx vitest run server/utils/tcgplayerCsv.test.js Expected: FAIL — Failed to resolve import "./tcgplayerCsv.js"
- [ ] Step 3: Write the implementation
Create server/utils/tcgplayerCsv.js:
javascript
/**
* TCGplayer CSV format module.
*
* Owns the vocabulary and shape of TCGplayer's exports. Two consumers:
* buylistQuoteService (collection export, condition helpers only) and
* tcgImportService (Pricing Custom Export, everything).
*
* Every literal here was read from a real Pricing Custom Export captured
* 2026-08-04 (8,514 rows) — see the design spec's "Source data" section.
* Nothing is assumed from TCGplayer documentation (CLAUDE.md §5.4).
*/
'use strict';
const VALID_CONDITIONS = ['nm', 'lp', 'mp', 'hp', 'damaged'];
// TCGplayer carries the finish inside Condition ("Near Mint Foil"). Its
// presence is the only finish signal the export has, so its absence on a
// populated Condition means nonfoil; a blank Condition tells us nothing.
const TCGPLAYER_FOIL_SUFFIX_RE = /\s*foil\s*$/i;
const CONDITION_ALIASES = {
'near mint': 'nm',
'lightly played': 'lp',
'moderately played': 'mp',
'heavily played': 'hp',
'damaged': 'damaged'
};
// The Condition value TCGplayer writes for sealed product. It is NOT in
// CONDITION_ALIASES on purpose: normalizeCondition would fold it to 'nm'
// and a booster box would import as a Near Mint single.
const SEALED_CONDITION = 'Unopened';
// The Product Line value for Magic, read from the reference export. Pokemon
// and Riftbound literals are unverified and deliberately absent — an unknown
// line is reported, never assumed (see the plan's Global Constraints).
const MTG_PRODUCT_LINE = 'Magic';
// Columns unique to the Pricing Custom Export. "TCGplayer Id" is NOT usable
// for detection: the collection export has it too.
const REQUIRED_COLUMNS = ['total quantity', 'tcg marketplace price'];
// Not the buylist paste guard (MAX_LINES = 500). The reference export alone
// is 8,514 rows; this is the abuse ceiling, ~12x that.
const MAX_IMPORT_ROWS = 100000;
class TcgImportError extends Error {
constructor(message, code) {
super(message);
this.name = 'TcgImportError';
this.code = code;
}
}
function stripFoilSuffix(condition) {
return String(condition || '').replace(TCGPLAYER_FOIL_SUFFIX_RE, '').trim();
}
function tcgplayerFinish(condition) {
const value = String(condition || '').trim();
if (!value) return null;
return TCGPLAYER_FOIL_SUFFIX_RE.test(value) ? 'foil' : 'nonfoil';
}
function normalizeCondition(raw) {
if (!raw) return 'nm';
// ManaBox writes underscores ("near_mint"); TCGplayer writes spaces.
// Folding underscores lets one alias table serve both.
const key = raw.toLowerCase().trim().replace(/_/g, ' ');
if (VALID_CONDITIONS.includes(key)) return key;
// hasOwnProperty guard: "__proto__" and "constructor" survive
// toLowerCase().trim() and would otherwise resolve through the prototype
// chain to non-string values instead of falling through to 'nm'.
return Object.prototype.hasOwnProperty.call(CONDITION_ALIASES, key)
? CONDITION_ALIASES[key] // eslint-disable-line security/detect-object-injection -- key only ever indexes the small hardcoded CONDITION_ALIASES map, guarded by hasOwnProperty above
: 'nm';
}
/**
* Header column names come from a real file, but csv-parser preserves the
* exporter's casing and any stray spaces. Index by lowercased/trimmed key so
* a cosmetic change upstream doesn't silently mismatch every column.
*/
function indexRowByLowercaseKey(row) {
const map = new Map();
for (const [key, value] of Object.entries(row || {})) {
map.set(String(key).toLowerCase().trim(), value);
}
return map;
}
function assertPricingExportHeader(headerLine) {
const columns = String(headerLine || '')
.split(',')
.map((c) => c.replace(/^"|"$/g, '').toLowerCase().trim());
const missing = REQUIRED_COLUMNS.filter((c) => !columns.includes(c));
if (missing.length) {
throw new TcgImportError(
'That file is not a TCGplayer Pricing Custom Export. In TCGplayer Seller Portal, ' +
'go to Pricing and export with the Total Quantity and TCG Marketplace Price columns included.',
'WRONG_TCGPLAYER_EXPORT'
);
}
}
/**
* Bucket a raw CSV row. First match wins; buckets are exclusive.
* Order is load-bearing — see the SEALED_CONDITION comment above.
*/
function classifyRow(row) {
const fields = indexRowByLowercaseKey(row);
const quantity = parseInt(fields.get('total quantity'), 10);
if (!Number.isFinite(quantity) || quantity <= 0) {
return { status: 'zero_qty', reason: 'No stock on TCGplayer', parsed: null };
}
const productLine = String(fields.get('product line') || '').trim();
if (productLine !== MTG_PRODUCT_LINE) {
return {
status: 'unsupported_line',
reason: `${productLine || 'Unknown'} isn't supported yet — this import handles Magic only`,
parsed: null
};
}
const rawCondition = String(fields.get('condition') || '').trim();
if (rawCondition === SEALED_CONDITION) {
return {
status: 'sealed',
reason: null,
parsed: { finish: null, condition: SEALED_CONDITION, quantity }
};
}
return {
status: 'single',
reason: null,
parsed: {
finish: tcgplayerFinish(rawCondition),
condition: normalizeCondition(stripFoilSuffix(rawCondition)),
quantity
}
};
}
module.exports = {
stripFoilSuffix,
tcgplayerFinish,
normalizeCondition,
indexRowByLowercaseKey,
assertPricingExportHeader,
classifyRow,
TcgImportError,
VALID_CONDITIONS,
SEALED_CONDITION,
MTG_PRODUCT_LINE,
MAX_IMPORT_ROWS
};- [ ] Step 4: Run the tests to verify they pass
Run: npx vitest run server/utils/tcgplayerCsv.test.js Expected: PASS, all cases.
- [ ] Step 5: Point buylistQuoteService at the new module
In server/services/buylistQuoteService.js, delete the local definitions of TCGPLAYER_FOIL_SUFFIX_RE, stripFoilSuffix, tcgplayerFinish, normalizeCondition, CONDITION_ALIASES, and VALID_CONDITIONS (around lines 19 and 111–143 and 342–359), and add near the other requires at the top:
javascript
const {
stripFoilSuffix,
tcgplayerFinish,
normalizeCondition,
VALID_CONDITIONS
} = require('../utils/tcgplayerCsv');Leave indexRowByLowercaseKey wherever buylistQuoteService currently defines it if it has one — do not remove a second definition in this task; if it already imports one, use the shared one. Every remaining call site (parseTcgplayerRow, parseManaBoxRow, parseCardList) keeps working unchanged because the signatures are identical.
- [ ] Step 6: Run the buylist tests to prove nothing regressed
Run: npx vitest run server/services/buylistQuoteService.test.js Expected: PASS — the existing suite is the regression net for this extraction. If any test fails, the extraction changed behavior; fix the extraction, not the test.
- [ ] Step 7: Lint and commit
bash
npm run lint
git add server/utils/tcgplayerCsv.js server/utils/tcgplayerCsv.test.js server/services/buylistQuoteService.js
git commit -m "Read TCGplayer export conditions from one shared module instead of two"Task 2: The TCGplayer set-name bridge
Resolve a TCGplayer Set Name to one of our MTG set codes, via TCGCSV group ids.
Files:
- Create:
server/services/tcgcsvGroupService.js - Create:
server/services/tcgcsvGroupService.test.js
Interfaces:
Consumes:
TcgImportErrorfromserver/utils/tcgplayerCsv.js(Task 1).Produces:
resolveSetCodes(setNames: string[]): Promise<Map<string, { groupId: number, setCode: string } | null>>— keys are the input names verbatim; anullvalue means unresolvable. Used by Task 4._setDeps(deps)/_resetDeps()wheredepsis{ fetchGroups, redis, SetModel }.
[ ] Step 1: Write the failing tests
Create server/services/tcgcsvGroupService.test.js:
javascript
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
import { resolveSetCodes, _setDeps, _resetDeps } from './tcgcsvGroupService.js';
// Three real rows from https://tcgcsv.com/tcgplayer/1/groups, read 2026-08-04.
const GROUPS = [
{ groupId: 1, name: '10th Edition', abbreviation: '10E' },
{ groupId: 2576, name: 'Secret Lair Drop Series', abbreviation: 'SLD' },
{ groupId: 23874, name: 'Art Series: Lorwyn Eclipsed', abbreviation: 'AECL' }
];
// Our side: mtg_sets docs. 'Art Series: Lorwyn Eclipsed' deliberately has no
// row — it is one of the 10 real sets that do not resolve (24 of 2,944 rows).
const SETS = [
{ data: { code: '10E', tcgplayerGroupId: 1 } },
{ data: { code: 'SLD', tcgplayerGroupId: 2576 } }
];
function stubs(overrides = {}) {
return {
fetchGroups: async () => GROUPS,
redis: null,
SetModel: {
find: () => ({ select: () => ({ lean: async () => SETS }) })
},
...overrides
};
}
beforeEach(() => _resetDeps());
afterEach(() => _resetDeps());
describe('resolveSetCodes', () => {
it('resolves a set name to its group id and our set code', async () => {
_setDeps(stubs());
const map = await resolveSetCodes(['10th Edition']);
expect(map.get('10th Edition')).toEqual({ groupId: 1, setCode: '10E' });
});
it('matches set names case-insensitively and ignoring surrounding space', async () => {
_setDeps(stubs());
const map = await resolveSetCodes([' 10TH EDITION ']);
expect(map.get(' 10TH EDITION ').setCode).toBe('10E');
});
it('returns null for a group we carry no set for, rather than guessing', async () => {
_setDeps(stubs());
const map = await resolveSetCodes(['Art Series: Lorwyn Eclipsed']);
expect(map.get('Art Series: Lorwyn Eclipsed')).toBeNull();
});
it('returns null for a set name TCGCSV does not know', async () => {
_setDeps(stubs());
const map = await resolveSetCodes(['Not A Real Set']);
expect(map.get('Not A Real Set')).toBeNull();
});
it('keys the result by the caller\'s exact input string', async () => {
_setDeps(stubs());
const map = await resolveSetCodes(['10th Edition', 'Not A Real Set']);
expect([...map.keys()]).toEqual(['10th Edition', 'Not A Real Set']);
});
it('fetches the group list once for many set names', async () => {
let calls = 0;
_setDeps(stubs({ fetchGroups: async () => { calls++; return GROUPS; } }));
await resolveSetCodes(['10th Edition', 'Secret Lair Drop Series']);
expect(calls).toBe(1);
});
it('serves the group list from Redis without fetching', async () => {
let calls = 0;
_setDeps(stubs({
fetchGroups: async () => { calls++; return GROUPS; },
redis: { get: async () => JSON.stringify(GROUPS), setex: async () => 'OK' }
}));
const map = await resolveSetCodes(['10th Edition']);
expect(calls).toBe(0);
expect(map.get('10th Edition').setCode).toBe('10E');
});
it('caches a fresh fetch into Redis for 24h', async () => {
let ttl = null;
_setDeps(stubs({
redis: { get: async () => null, setex: async (_k, seconds) => { ttl = seconds; return 'OK'; } }
}));
await resolveSetCodes(['10th Edition']);
expect(ttl).toBe(86400);
});
it('surfaces a TCGCSV outage as a clear error rather than matching on set names', async () => {
_setDeps(stubs({ fetchGroups: async () => { throw new Error('socket hang up'); } }));
await expect(resolveSetCodes(['10th Edition'])).rejects.toMatchObject({
name: 'TcgImportError',
code: 'TCGCSV_UNAVAILABLE'
});
});
});- [ ] Step 2: Run the tests to verify they fail
Run: npx vitest run server/services/tcgcsvGroupService.test.js Expected: FAIL — Failed to resolve import "./tcgcsvGroupService.js"
- [ ] Step 3: Write the implementation
Create server/services/tcgcsvGroupService.js:
javascript
/**
* TCGplayer set-name bridge.
*
* A TCGplayer export names sets the way TCGplayer does ("10th Edition");
* MTGJSON names them its own way ("Tenth Edition"). The two are not
* interchangeable, so nothing here ever compares set *names* across the two
* sources. The join runs through TCGplayer's own group ids, which MTGJSON
* records as `tcgplayerGroupId`:
*
* Set Name -> TCGCSV group name -> groupId -> SetModel.data.tcgplayerGroupId -> code
*
* Measured against the reference export 2026-08-04: 319/319 set names match a
* TCGCSV group exactly, and 309/319 resolve all the way to a set code. The 10
* that don't (Art Series, Promo Packs, Mystery Booster 2 Playtest, Special
* Occasion) are 24 of 2,944 stocked rows.
*/
'use strict';
const https = require('https');
const { getRedisConnection } = require('../config/redis');
const SetModel = require('../models/SetModel');
const { TcgImportError } = require('../utils/tcgplayerCsv');
const logger = require('../utils/logger');
const TCGCSV_GROUPS_URL = 'https://tcgcsv.com/tcgplayer/1/groups';
const CACHE_KEY = 'tcgcsv:mtg:groups:v1';
const CACHE_TTL_SECONDS = 86400; // 24h — TCGCSV publishes daily.
// TCGCSV blocks unidentified agents with a plaintext message (not JSON), which
// then fails JSON.parse with a confusing "Unexpected token 'A'". Verified
// 2026-08-04. Same UA the Pokemon importer already sends.
const REQUEST_OPTIONS = {
headers: { 'User-Agent': 'lgs-forge/1.0 (ufkesba@gmail.com)' }
};
function httpsGetJSON(url) {
return new Promise((resolve, reject) => {
https.get(url, REQUEST_OPTIONS, (res) => {
if (res.statusCode >= 300 && res.statusCode < 400 && res.headers.location) {
res.resume();
return httpsGetJSON(res.headers.location).then(resolve, reject);
}
if (res.statusCode !== 200) {
res.resume();
return reject(new Error(`TCGCSV returned HTTP ${res.statusCode}`));
}
let body = '';
res.on('data', (chunk) => { body += chunk; });
res.on('end', () => {
try { resolve(JSON.parse(body)); }
catch (err) { reject(new Error(`TCGCSV returned unparseable body: ${err.message}`)); }
});
}).on('error', reject);
});
}
async function defaultFetchGroups() {
const payload = await httpsGetJSON(TCGCSV_GROUPS_URL);
return payload.results || [];
}
let _deps = null;
function _setDeps(deps) { _deps = deps; }
function _resetDeps() { _deps = null; }
function getDeps() {
if (_deps) return _deps;
let redis = null;
try { redis = getRedisConnection(); } catch { redis = null; }
return { fetchGroups: defaultFetchGroups, redis, SetModel };
}
function normalizeName(name) {
return String(name || '').toLowerCase().trim();
}
async function loadGroups(deps) {
if (deps.redis) {
try {
const cached = await deps.redis.get(CACHE_KEY);
if (cached) return JSON.parse(cached);
} catch (err) {
logger.warn('TCGCSV group cache read failed; fetching live', { error: err.message });
}
}
let groups;
try {
groups = await deps.fetchGroups();
} catch (err) {
// Deliberately no fallback to MTGJSON set-name matching: TCGplayer says
// "10th Edition" where MTGJSON says "Tenth Edition", so a name guess
// mismatches silently instead of failing (CLAUDE.md §5.4).
throw new TcgImportError(
"We couldn't reach TCGplayer's set list just now. Try the upload again in a few minutes.",
'TCGCSV_UNAVAILABLE'
);
}
if (deps.redis) {
try { await deps.redis.setex(CACHE_KEY, CACHE_TTL_SECONDS, JSON.stringify(groups)); }
catch (err) { logger.warn('TCGCSV group cache write failed', { error: err.message }); }
}
return groups;
}
/**
* @param {string[]} setNames - TCGplayer set names, verbatim from the CSV.
* @returns {Promise<Map<string, {groupId:number, setCode:string}|null>>}
* Keyed by the caller's exact input string; null means unresolvable.
*/
async function resolveSetCodes(setNames) {
const deps = getDeps();
const groups = await loadGroups(deps);
const groupByName = new Map(groups.map((g) => [normalizeName(g.name), g]));
const wanted = [];
const groupIdByInput = new Map();
for (const name of setNames) {
const group = groupByName.get(normalizeName(name));
if (group) {
groupIdByInput.set(name, group.groupId);
wanted.push(group.groupId);
} else {
groupIdByInput.set(name, null);
}
}
const sets = wanted.length
? await deps.SetModel.find({ 'data.tcgplayerGroupId': { $in: wanted } })
.select('data.code data.tcgplayerGroupId')
.lean()
: [];
const codeByGroupId = new Map(sets.map((s) => [s.data.tcgplayerGroupId, s.data.code]));
const out = new Map();
for (const name of setNames) {
const groupId = groupIdByInput.get(name);
const setCode = groupId == null ? null : codeByGroupId.get(groupId);
out.set(name, setCode ? { groupId, setCode } : null);
}
return out;
}
module.exports = { resolveSetCodes, _setDeps, _resetDeps };- [ ] Step 4: Run the tests to verify they pass
Run: npx vitest run server/services/tcgcsvGroupService.test.js Expected: PASS, all cases.
- [ ] Step 5: Verify the bridge against the live source and our own data
This step exists because the design's 309/319 figure was measured against MTGJSON's SetList.json upstream, not against our mtg_sets collection. Confirm our copy carries tcgplayerGroupId.
bash
docker compose up -d
npm run data:update-setsThen run a one-off check (delete the file afterwards — it is not part of the diff):
bash
node -e "
const m=require('mongoose');
(async()=>{
await m.connect('mongodb://localhost:27017/lgs-ledger');
const c=m.connection.db.collection('mtg_sets');
const total=await c.countDocuments();
const withGid=await c.countDocuments({'data.tcgplayerGroupId':{\$exists:true,\$ne:null}});
console.log('mtg_sets:',total,'with tcgplayerGroupId:',withGid);
process.exit(0);
})();"Expected: with tcgplayerGroupId is within a few percent of total. If it is near zero, our importer is dropping the field and that is a blocker — stop and report it rather than continuing; every later task depends on this join.
- [ ] Step 6: Lint and commit
bash
npm run lint
git add server/services/tcgcsvGroupService.js server/services/tcgcsvGroupService.test.js
git commit -m "Resolve TCGplayer set names to our set codes through TCGplayer group ids"Task 3: The import and import-line models
Files:
- Create:
server/models/TcgImport.js - Create:
server/models/TcgImport.test.js - Create:
server/models/TcgImportLine.js - Create:
server/models/TcgImportLine.test.js
Interfaces:
Produces: Mongoose models
TcgImport(collectiontcg_imports) andTcgImportLine(collectiontcg_import_lines). Field names as written below; Tasks 4 and 5 read and write exactly these.[ ] Step 1: Write the failing round-trip tests
These are §5.2 guard tests: each one fails if a schema line is deleted, because Mongoose strict mode silently drops undeclared sub-document fields on write.
Create server/models/TcgImport.test.js:
javascript
import { describe, it, expect } from 'vitest';
import TcgImport from './TcgImport.js';
const validDoc = {
shop: 'test-store.myshopify.com',
game: 'mtg',
status: 'preview',
filename: 'TCGplayer__Pricing_Custom_Export_20260804_020845.csv',
uploadedAt: new Date('2026-08-04T02:08:45Z'),
uploadedBy: 'ufkesba@gmail.com',
priceMode: 'sync',
counts: {
totalRows: 8514, zeroQty: 5570, matched: 2920, unmatched: 24,
sealed: 0, distinctProducts: 2829, distinctSets: 319, units: 4082
},
tierCheck: {
tier: 'free', limit: 500, current: 0, estimated: 2829,
allowed: false, requiredTier: 'rampUp', exact: false
},
setMap: [{ tcgSetName: '10th Edition', groupId: 1, setCode: '10E' }],
progress: {
phase: 'preview', setsDone: [], productsCreated: 0, productsUpdated: 0,
variantsCreated: 0, inventorySet: 0, failed: 0
}
};
describe('TcgImport schema', () => {
it('round-trips every nested counts field (fails if a schema line is deleted)', () => {
const doc = new TcgImport(validDoc);
expect(doc.counts.totalRows).toBe(8514);
expect(doc.counts.zeroQty).toBe(5570);
expect(doc.counts.matched).toBe(2920);
expect(doc.counts.unmatched).toBe(24);
expect(doc.counts.sealed).toBe(0);
expect(doc.counts.distinctProducts).toBe(2829);
expect(doc.counts.distinctSets).toBe(319);
expect(doc.counts.units).toBe(4082);
});
it('round-trips every nested tierCheck field, including exact', () => {
const doc = new TcgImport(validDoc);
expect(doc.tierCheck.tier).toBe('free');
expect(doc.tierCheck.limit).toBe(500);
expect(doc.tierCheck.current).toBe(0);
expect(doc.tierCheck.estimated).toBe(2829);
expect(doc.tierCheck.allowed).toBe(false);
expect(doc.tierCheck.requiredTier).toBe('rampUp');
expect(doc.tierCheck.exact).toBe(false);
});
it('round-trips every nested progress field', () => {
const doc = new TcgImport(validDoc);
expect(doc.progress.setsDone).toEqual([]);
expect(doc.progress.productsCreated).toBe(0);
expect(doc.progress.productsUpdated).toBe(0);
expect(doc.progress.variantsCreated).toBe(0);
expect(doc.progress.inventorySet).toBe(0);
expect(doc.progress.failed).toBe(0);
});
it('round-trips the setMap entries', () => {
const doc = new TcgImport(validDoc);
expect(doc.setMap[0].tcgSetName).toBe('10th Edition');
expect(doc.setMap[0].groupId).toBe(1);
expect(doc.setMap[0].setCode).toBe('10E');
});
it('requires shop and game — game is never defaulted', () => {
expect(new TcgImport({ ...validDoc, shop: undefined }).validateSync().errors.shop).toBeDefined();
expect(new TcgImport({ ...validDoc, game: undefined }).validateSync().errors.game).toBeDefined();
});
it('rejects a status outside the declared set, including cancelled', () => {
expect(new TcgImport({ ...validDoc, status: 'cancelled' }).validateSync().errors.status).toBeDefined();
});
it('rejects a priceMode outside sync and locked', () => {
expect(new TcgImport({ ...validDoc, priceMode: 'whatever' }).validateSync().errors.priceMode).toBeDefined();
});
});Create server/models/TcgImportLine.test.js:
javascript
import { describe, it, expect } from 'vitest';
import mongoose from 'mongoose';
import TcgImportLine from './TcgImportLine.js';
const validDoc = {
importId: new mongoose.Types.ObjectId(),
shop: 'test-store.myshopify.com',
raw: {
tcgId: '376073', productLine: 'Magic', setName: '10th Edition',
productName: 'Adarkar Wastes', number: '347', rarity: 'R',
condition: 'Lightly Played', totalQuantity: 1,
marketplacePrice: 5.37, marketPrice: 4.63
},
parsed: { finish: 'nonfoil', condition: 'lp', quantity: 1 },
match: {
status: 'matched', reason: null, setCode: '10E',
cardUuid: 'abc-123', collectorNumber: '347', rarity: 'rare', finish: 'nonfoil'
},
result: { productId: null, variantId: null, action: null, error: null }
};
describe('TcgImportLine schema', () => {
it('round-trips every raw field (fails if a schema line is deleted)', () => {
const doc = new TcgImportLine(validDoc);
expect(doc.raw.tcgId).toBe('376073');
expect(doc.raw.productLine).toBe('Magic');
expect(doc.raw.setName).toBe('10th Edition');
expect(doc.raw.productName).toBe('Adarkar Wastes');
expect(doc.raw.number).toBe('347');
expect(doc.raw.rarity).toBe('R');
expect(doc.raw.condition).toBe('Lightly Played');
expect(doc.raw.totalQuantity).toBe(1);
expect(doc.raw.marketplacePrice).toBe(5.37);
expect(doc.raw.marketPrice).toBe(4.63);
});
it('keeps raw.tcgId a string — it is a SKU id, never a number to join on', () => {
expect(typeof new TcgImportLine(validDoc).raw.tcgId).toBe('string');
});
it('round-trips every parsed and match field', () => {
const doc = new TcgImportLine(validDoc);
expect(doc.parsed.finish).toBe('nonfoil');
expect(doc.parsed.condition).toBe('lp');
expect(doc.parsed.quantity).toBe(1);
expect(doc.match.status).toBe('matched');
expect(doc.match.setCode).toBe('10E');
expect(doc.match.cardUuid).toBe('abc-123');
expect(doc.match.collectorNumber).toBe('347');
expect(doc.match.rarity).toBe('rare');
expect(doc.match.finish).toBe('nonfoil');
});
it('round-trips the result fields PR 2 writes into', () => {
const doc = new TcgImportLine({
...validDoc,
result: { productId: 'gid://shopify/Product/1', variantId: 'gid://shopify/ProductVariant/2', action: 'created', error: null }
});
expect(doc.result.productId).toBe('gid://shopify/Product/1');
expect(doc.result.variantId).toBe('gid://shopify/ProductVariant/2');
expect(doc.result.action).toBe('created');
});
it('accepts every declared match status', () => {
for (const status of ['matched', 'unmatched', 'zero_qty', 'unsupported_line', 'sealed']) {
const doc = new TcgImportLine({ ...validDoc, match: { ...validDoc.match, status } });
expect(doc.validateSync()?.errors?.['match.status']).toBeUndefined();
}
});
it('rejects an undeclared match status', () => {
const doc = new TcgImportLine({ ...validDoc, match: { ...validDoc.match, status: 'maybe' } });
expect(doc.validateSync().errors['match.status']).toBeDefined();
});
it('requires importId and shop', () => {
expect(new TcgImportLine({ ...validDoc, importId: undefined }).validateSync().errors.importId).toBeDefined();
expect(new TcgImportLine({ ...validDoc, shop: undefined }).validateSync().errors.shop).toBeDefined();
});
});- [ ] Step 2: Run the tests to verify they fail
Run: npx vitest run server/models/TcgImport.test.js server/models/TcgImportLine.test.js Expected: FAIL — both imports unresolved.
- [ ] Step 3: Write TcgImport
Create server/models/TcgImport.js:
javascript
/**
* TCGplayer Import
*
* One document per uploaded TCGplayer Pricing Custom Export, per store.
* Holds the analysis, the merchant's decisions, and the execution checkpoint.
* The rows themselves live in tcg_import_lines — see that model for why.
*
* Every sub-document field is declared field-by-field: Mongoose strict mode
* silently drops undeclared nested fields on write (CLAUDE.md §5.2).
*/
const mongoose = require('mongoose');
const tcgImportSchema = new mongoose.Schema({
shop: { type: String, required: true, index: true },
// Explicit, never defaulted (§5.5). MTG-only in this arc, but the field
// exists from the start so Pokemon needs no migration.
game: { type: String, required: true },
// No 'cancelled': cancellation is out of scope for this arc, and an
// unreachable status is dead scaffolding (§5.9).
status: {
type: String,
required: true,
enum: ['preview', 'queued', 'running', 'completed', 'failed'],
default: 'preview'
},
filename: String,
uploadedAt: { type: Date, default: Date.now },
uploadedBy: String,
// Set at confirm, not at upload. 'sync' = our computed pricing, updated
// daily. 'locked' = the file's TCG Marketplace Price, held until the
// merchant clears the price_locked metafield (PR 3).
priceMode: { type: String, enum: ['sync', 'locked'] },
// One counter per classifyRow bucket. The buckets are exclusive and
// exhaustive, so totalRows must equal
// zeroQty + sealed + matched + unmatched + unsupportedLine — assert that in
// the bucket test. Without its own counter an unsupported_line row is
// counted nowhere AND is absent from unmatchedSample, so it disappears from
// the preview entirely and the merchant sees a total that does not add up.
counts: {
totalRows: Number,
zeroQty: Number,
matched: Number,
unmatched: Number,
unsupportedLine: Number,
sealed: Number,
distinctProducts: Number,
distinctSets: Number,
units: Number
},
tierCheck: {
tier: String,
limit: Number,
current: Number,
estimated: Number,
allowed: Boolean,
requiredTier: String,
// false = conservative estimate (already-synced cards not excluded);
// true = a cache build confirmed the real new-product count.
exact: { type: Boolean, default: false }
},
// The resolved bridge, kept so a re-run needs no second TCGCSV fetch and
// so a wrong match can be traced back to the group it came from.
setMap: [{
_id: false,
tcgSetName: String,
groupId: Number,
setCode: String
}],
progress: {
phase: String,
// Resume checkpoint: set codes already finished. A restarted worker
// re-enters at the next unfinished set (PR 2).
setsDone: [String],
productsCreated: { type: Number, default: 0 },
productsUpdated: { type: Number, default: 0 },
variantsCreated: { type: Number, default: 0 },
inventorySet: { type: Number, default: 0 },
failed: { type: Number, default: 0 }
},
summary: String,
error: String
}, { timestamps: true });
// The import list for one store, newest first.
tcgImportSchema.index({ shop: 1, createdAt: -1 });
module.exports = mongoose.model('TcgImport', tcgImportSchema, 'tcg_imports');- [ ] Step 4: Write TcgImportLine
Create server/models/TcgImportLine.js:
javascript
/**
* TCGplayer Import Line
*
* One document per CSV row. Deliberately a separate collection rather than an
* array on TcgImport: the reference export's 2,944 stocked rows embed to
* roughly 600KB, but a 50k-row merchant would push a single document toward
* Mongo's 16MB ceiling. §5.6's lesson is that materialising an unbounded set
* is a bug even when nothing errors today.
*
* The raw row is stored verbatim so the unmatched report can show a merchant
* their own line, and so a re-run needs no re-upload.
*/
const mongoose = require('mongoose');
const tcgImportLineSchema = new mongoose.Schema({
importId: { type: mongoose.Schema.Types.ObjectId, ref: 'TcgImport', required: true },
shop: { type: String, required: true },
raw: {
// SKU-level, NOT a product id: measured 1,403/1,403 multi-condition
// groups carry a different id per condition. Kept for diagnostics and
// joined on by nothing. String, because it is an identifier.
tcgId: String,
productLine: String,
setName: String,
productName: String,
number: String,
// TCGplayer's own vocabulary (R/M/U/C/P/S/L/T). Never mapped to our
// rarity — match.rarity comes from the catalog document (§5.4).
rarity: String,
condition: String,
totalQuantity: Number,
marketplacePrice: Number,
marketPrice: Number
},
parsed: {
finish: String,
condition: String,
quantity: Number
},
match: {
status: {
type: String,
required: true,
enum: ['matched', 'unmatched', 'zero_qty', 'unsupported_line', 'sealed']
},
// Plain language, shown to the merchant verbatim.
reason: String,
setCode: String,
cardUuid: String,
collectorNumber: String,
rarity: String,
finish: String
},
// Written by PR 2's processor; declared now so the shape is fixed.
result: {
productId: String,
variantId: String,
action: String,
error: String
}
}, { timestamps: true });
// The processor reads one bounded set at a time; the report paginates by status.
// Both paths are dotted: the set code lives at match.setCode, NOT at the top
// level. An index on a bare `setCode` would cover a path no document has, index
// every row as null, and leave the per-set read scanning the whole import.
tcgImportLineSchema.index({ importId: 1, 'match.setCode': 1 });
tcgImportLineSchema.index({ importId: 1, 'match.status': 1 });
module.exports = mongoose.model('TcgImportLine', tcgImportLineSchema, 'tcg_import_lines');- [ ] Step 5: Run the tests to verify they pass
Run: npx vitest run server/models/TcgImport.test.js server/models/TcgImportLine.test.js Expected: PASS, all cases.
- [ ] Step 6: Lint and commit
bash
npm run lint
git add server/models/TcgImport.js server/models/TcgImport.test.js server/models/TcgImportLine.js server/models/TcgImportLine.test.js
git commit -m "Store an uploaded TCGplayer export and its rows for review"Task 4: The import service — parse, match, persist
Files:
- Create:
server/services/tcgImportService.js - Create:
server/services/tcgImportService.test.js - Create:
server/services/__fixtures__/tcgplayer-pricing-export-sample.csv
Interfaces:
Consumes:
classifyRow,assertPricingExportHeader,TcgImportError,MAX_IMPORT_ROWS(Task 1);resolveSetCodes(Task 2);TcgImport,TcgImportLine(Task 3).Produces:
buildPreview({ shop, game, csvText, filename, uploadedBy }): Promise<{ importId, counts, setMap, unmatchedSample }>— persists oneTcgImportand itsTcgImportLinerows. Used by Task 5.matchLine({ setCode, productName, number, finish }, game): Promise<match>— exported for testing and reused by PR 2._setDeps(deps)/_resetDeps()wheredepsis{ resolveSetCodes, getPlugin, TcgImport, TcgImportLine }.
[ ] Step 1: Create the test fixture
Create server/services/__fixtures__/tcgplayer-pricing-export-sample.csv. Column names, Product Line, Rarity letters and Condition vocabulary are verbatim from the real export; quantities and prices are synthesized so no live business data enters the repo. Every row exists to exercise one branch.
csv
TCGplayer Id,Product Line,Set Name,Product Name,Title,Number,Rarity,Condition,TCG Market Price,TCG Direct Low,TCG Low Price With Shipping,TCG Low Price,Total Quantity,Add to Quantity,TCG Marketplace Price,My Store Reserve Quantity,My Store Price,Photo URL
"376073","Magic","10th Edition","Adarkar Wastes","","347","R","Lightly Played","4.63","","5.3800","3.8900","2","0","5.3700","","",""
"790377","Magic","10th Edition","Civic Wayfinder","","255","C","Moderately Played Foil","0.40","","1.9400","0.4400","1","0","1.9000","","",""
"4951","Magic","10th Edition","Coat of Arms","","316","R","Near Mint","15.85","","17.9900","16.0000","0","0","16.0100","","",""
"344601","Magic","Alliances","Lodestone Bauble","","","R","Moderately Played","3.10","","4.0000","3.5000","3","0","4.0000","","",""
"999001","Magic","10th Edition","Forest","","383","L","Lightly Played","0.20","","0.5000","0.1000","4","0","0.4900","","",""
"7488026","Magic","Secret Lair Drop Series","Secret Lair Drop: Calling All Hydra Heads (WPN Exclusive) - Traditional Foil Edition","","","","Unopened","30.66","","33.0000","30.0000","1","0","30.6600","","",""
"888001","Pokemon","Base Set","Charizard","","4","R","Near Mint","250.00","","300.0000","240.0000","1","0","299.0000","","",""
"999002","Magic","Art Series: Lorwyn Eclipsed","Ancient Grudge","","5","T","Damaged Foil","1.00","","2.0000","0.9000","1","0","1.9500","","",""- [ ] Step 2: Write the failing tests
Create server/services/tcgImportService.test.js:
javascript
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
import fs from 'fs';
import path from 'path';
import { buildPreview, matchLine, _setDeps, _resetDeps } from './tcgImportService.js';
const CSV = fs.readFileSync(
path.join(process.cwd(), 'server/services/__fixtures__/tcgplayer-pricing-export-sample.csv'),
'utf8'
);
// Stand-in catalog: one doc per (set_code, collector_number). 'Forest' appears
// twice in 10E with different numbers, so a name-only lookup is ambiguous.
const CATALOG = [
{ sourceCardUUID: 'u-adarkar', metafields: { set_code: '10E', collector_number: '347', card_name: 'Adarkar Wastes', rarity: 'Rare' }, variants: [{ finish: 'nonfoil' }, { finish: 'foil' }] },
{ sourceCardUUID: 'u-civic', metafields: { set_code: '10E', collector_number: '255', card_name: 'Civic Wayfinder', rarity: 'Common' }, variants: [{ finish: 'nonfoil' }, { finish: 'foil' }] },
{ sourceCardUUID: 'u-forest1', metafields: { set_code: '10E', collector_number: '383', card_name: 'Forest', rarity: 'Land' }, variants: [{ finish: 'nonfoil' }] },
{ sourceCardUUID: 'u-forest2', metafields: { set_code: '10E', collector_number: '384', card_name: 'Forest', rarity: 'Land' }, variants: [{ finish: 'nonfoil' }] },
{ sourceCardUUID: 'u-bauble', metafields: { set_code: 'ALL', collector_number: '112', card_name: 'Lodestone Bauble', rarity: 'Rare' }, variants: [{ finish: 'nonfoil' }] }
];
function fakeModel(docs) {
const matches = (doc, q) => Object.entries(q).every(([k, v]) => {
const actual = k.split('.').reduce((o, part) => (o == null ? o : o[part]), doc);
return v instanceof RegExp ? v.test(actual) : actual === v;
});
return {
findOne: (q) => ({ lean: async () => docs.find((d) => matches(d, q)) || null }),
find: (q) => ({ lean: async () => docs.filter((d) => matches(d, q)) })
};
}
const saved = { imports: [], lines: [] };
function stubs(overrides = {}) {
return {
resolveSetCodes: async (names) => new Map(names.map((n) => [
n,
n === '10th Edition' ? { groupId: 1, setCode: '10E' }
: n === 'Alliances' ? { groupId: 8, setCode: 'ALL' }
: n === 'Secret Lair Drop Series' ? { groupId: 2576, setCode: 'SLD' }
: null
])),
getPlugin: () => ({
getProductModel: () => fakeModel(CATALOG),
selectVariantByFinish: (variants, finish) => variants.find((v) => v.finish === finish) || null
}),
TcgImport: {
create: async (doc) => { const d = { ...doc, _id: 'imp-1' }; saved.imports.push(d); return d; },
findByIdAndUpdate: async (_id, update) => { saved.imports[0] = { ...saved.imports[0], ...update.$set }; return saved.imports[0]; }
},
TcgImportLine: {
insertMany: async (docs) => { saved.lines.push(...docs); return docs; }
},
...overrides
};
}
beforeEach(() => { saved.imports = []; saved.lines = []; _resetDeps(); });
afterEach(() => _resetDeps());
describe('buildPreview', () => {
it('rejects a file that is not a Pricing Custom Export', async () => {
_setDeps(stubs());
await expect(buildPreview({
shop: 's.myshopify.com', game: 'mtg', csvText: 'name,qty\nFoo,1\n', filename: 'x.csv'
})).rejects.toMatchObject({ code: 'WRONG_TCGPLAYER_EXPORT' });
});
it('rejects a file over the row cap without parsing all of it', async () => {
_setDeps(stubs());
const header = CSV.split('\n')[0];
const body = new Array(100001).fill(CSV.split('\n')[1]).join('\n');
await expect(buildPreview({
shop: 's.myshopify.com', game: 'mtg', csvText: `${header}\n${body}`, filename: 'big.csv'
})).rejects.toMatchObject({ code: 'TOO_MANY_ROWS' });
});
it('counts every bucket from the sample file', async () => {
_setDeps(stubs());
const { counts } = await buildPreview({
shop: 's.myshopify.com', game: 'mtg', csvText: CSV, filename: 'sample.csv'
});
expect(counts.totalRows).toBe(8);
expect(counts.zeroQty).toBe(1); // Coat of Arms
expect(counts.sealed).toBe(1); // the Secret Lair Unopened row
expect(counts.matched).toBe(3); // Adarkar, Civic Wayfinder, Lodestone Bauble
expect(counts.unmatched).toBe(2); // ambiguous Forest + unresolvable Art Series
expect(counts.units).toBe(6); // 2 + 1 + 3 across matched rows only
});
it('does not count zero-quantity rows toward units or products', async () => {
_setDeps(stubs());
const { counts } = await buildPreview({
shop: 's.myshopify.com', game: 'mtg', csvText: CSV, filename: 'sample.csv'
});
expect(counts.distinctProducts).toBe(3);
});
it('buckets the Pokemon row as unsupported, not as MTG', async () => {
_setDeps(stubs());
await buildPreview({ shop: 's.myshopify.com', game: 'mtg', csvText: CSV, filename: 'sample.csv' });
const pokemon = saved.lines.find((l) => l.raw.productName === 'Charizard');
expect(pokemon.match.status).toBe('unsupported_line');
expect(pokemon.match.setCode).toBeNull();
});
it('persists the raw row verbatim so the report can quote it back', async () => {
_setDeps(stubs());
await buildPreview({ shop: 's.myshopify.com', game: 'mtg', csvText: CSV, filename: 'sample.csv' });
const line = saved.lines.find((l) => l.raw.productName === 'Adarkar Wastes');
expect(line.raw.tcgId).toBe('376073');
expect(line.raw.condition).toBe('Lightly Played');
expect(line.raw.rarity).toBe('R');
expect(line.raw.marketplacePrice).toBe(5.37);
});
it('records the resolved set map on the import', async () => {
_setDeps(stubs());
await buildPreview({ shop: 's.myshopify.com', game: 'mtg', csvText: CSV, filename: 'sample.csv' });
expect(saved.imports[0].setMap).toContainEqual({ tcgSetName: '10th Edition', groupId: 1, setCode: '10E' });
});
it('never writes a game it was not given', async () => {
_setDeps(stubs());
await buildPreview({ shop: 's.myshopify.com', game: 'mtg', csvText: CSV, filename: 'sample.csv' });
expect(saved.imports[0].game).toBe('mtg');
});
});
describe('matchLine', () => {
it('matches on set code plus collector number', async () => {
_setDeps(stubs());
const m = await matchLine({ setCode: '10E', productName: 'Adarkar Wastes', number: '347', finish: 'nonfoil' }, 'mtg');
expect(m.status).toBe('matched');
expect(m.cardUuid).toBe('u-adarkar');
});
it('takes rarity from our catalog, lowercased — never from the CSV letter', async () => {
_setDeps(stubs());
const m = await matchLine({ setCode: '10E', productName: 'Adarkar Wastes', number: '347', finish: 'nonfoil' }, 'mtg');
expect(m.rarity).toBe('rare');
expect(m.rarity).not.toBe('R');
});
it('falls back to name within the set when Number is blank', async () => {
// 196 of the reference export's stocked rows have no Number.
_setDeps(stubs());
const m = await matchLine({ setCode: 'ALL', productName: 'Lodestone Bauble', number: '', finish: 'nonfoil' }, 'mtg');
expect(m.status).toBe('matched');
expect(m.cardUuid).toBe('u-bauble');
});
it('reports an ambiguous name as unmatched instead of picking one', async () => {
_setDeps(stubs());
const m = await matchLine({ setCode: '10E', productName: 'Forest', number: '', finish: 'nonfoil' }, 'mtg');
expect(m.status).toBe('unmatched');
expect(m.reason).toMatch(/2 cards named Forest/i);
});
it('never matches a name outside the resolved set', async () => {
// Guard against the global findOne fallback: 'Lodestone Bauble' exists
// in ALL but must not be returned when the line resolved to 10E.
_setDeps(stubs());
const m = await matchLine({ setCode: '10E', productName: 'Lodestone Bauble', number: '', finish: 'nonfoil' }, 'mtg');
expect(m.status).toBe('unmatched');
});
it('reports an unresolvable set in plain language', async () => {
_setDeps(stubs());
const m = await matchLine({ setCode: null, productName: 'Ancient Grudge', number: '5', finish: 'foil', setName: 'Art Series: Lorwyn Eclipsed' }, 'mtg');
expect(m.status).toBe('unmatched');
expect(m.reason).toMatch(/Art Series: Lorwyn Eclipsed/);
});
it('resolves the finish through the plugin rather than trusting the CSV token', async () => {
_setDeps(stubs());
const m = await matchLine({ setCode: '10E', productName: 'Civic Wayfinder', number: '255', finish: 'foil' }, 'mtg');
expect(m.finish).toBe('foil');
});
it('falls back to the first variant when the asked-for finish is not printed', async () => {
_setDeps(stubs());
const m = await matchLine({ setCode: '10E', productName: 'Forest', number: '383', finish: 'foil' }, 'mtg');
expect(m.finish).toBe('nonfoil');
});
});- [ ] Step 3: Run the tests to verify they fail
Run: npx vitest run server/services/tcgImportService.test.js Expected: FAIL — Failed to resolve import "./tcgImportService.js"
- [ ] Step 4: Write the implementation
Create server/services/tcgImportService.js:
javascript
/**
* TCGplayer import — analysis half.
*
* Turns an uploaded Pricing Custom Export into a persisted, reviewable report.
* Writes nothing to Shopify: that is PR 2.
*
* The matcher here is deliberately NOT buylistQuoteService.matchCatalogLine.
* That one falls back to a global findOne on card name with no set constraint,
* which for a 2,900-row import would match cards to the wrong sets silently.
* Every lookup below is scoped to the resolved set code.
*/
'use strict';
const { Readable } = require('stream');
const csv = require('csv-parser');
const { escapeRegex } = require('../utils/escapeRegex');
const {
assertPricingExportHeader,
classifyRow,
indexRowByLowercaseKey,
TcgImportError,
MAX_IMPORT_ROWS
} = require('../utils/tcgplayerCsv');
const defaultGroupService = require('./tcgcsvGroupService');
const defaultPlugins = require('../plugins');
const TcgImport = require('../models/TcgImport');
const TcgImportLine = require('../models/TcgImportLine');
// Batched so a 100k-row file never holds every line document at once.
const INSERT_BATCH_SIZE = 1000;
const UNMATCHED_SAMPLE_SIZE = 50;
let _deps = null;
function _setDeps(deps) { _deps = deps; }
function _resetDeps() { _deps = null; }
function getDeps() {
if (_deps) return _deps;
return {
resolveSetCodes: defaultGroupService.resolveSetCodes,
getPlugin: defaultPlugins.getPlugin,
TcgImport,
TcgImportLine
};
}
function parseCsvRows(csvText) {
return new Promise((resolve, reject) => {
const rows = [];
Readable.from([csvText])
.pipe(csv())
.on('data', (row) => {
if (rows.length > MAX_IMPORT_ROWS) return; // hard stop, checked again below
rows.push(row);
})
.on('end', () => resolve(rows))
.on('error', reject);
});
}
function toNumber(value) {
const n = parseFloat(value);
return Number.isFinite(n) ? n : null;
}
/**
* Look a line up in the catalog, always scoped to its resolved set.
* @returns {Promise<object>} a `match` sub-document
*/
async function matchLine({ setCode, productName, number, finish, setName }, game, deps = getDeps()) {
if (!setCode) {
return {
status: 'unmatched',
reason: `We don't carry ${setName || 'that set'} yet`,
setCode: null, cardUuid: null, collectorNumber: null, rarity: null, finish: null
};
}
const plugin = deps.getPlugin(game);
const Model = plugin.getProductModel();
let doc = null;
if (number) {
doc = await Model.findOne({
'metafields.set_code': setCode,
'metafields.collector_number': number
}).lean();
}
if (!doc) {
// Name fallback, SCOPED TO THE SET. 196 of the reference export's
// stocked rows have no Number at all, so this path is load-bearing —
// but an unscoped version would match the wrong set's printing.
const byName = await Model.find({
'metafields.set_code': setCode,
'metafields.card_name': new RegExp(`^${escapeRegex(productName)}$`, 'i')
}).lean();
if (byName.length > 1) {
return {
status: 'unmatched',
reason: `${byName.length} cards named ${productName} in ${setCode} — we can't tell which one you have`,
setCode, cardUuid: null, collectorNumber: null, rarity: null, finish: null
};
}
doc = byName[0] || null;
}
if (!doc) {
return {
status: 'unmatched',
reason: `We don't have ${productName} in ${setCode}`,
setCode, cardUuid: null, collectorNumber: null, rarity: null, finish: null
};
}
// Rarity always comes from our catalog. The CSV's R/M/U/C/P/S/L/T is
// TCGplayer's vocabulary and is never mapped (§5.4).
const rarity = (doc.metafields.rarity || 'common').toLowerCase();
const variant = plugin.selectVariantByFinish(doc.variants, finish || null) || doc.variants[0];
return {
status: 'matched',
reason: null,
setCode,
cardUuid: doc.sourceCardUUID || doc.sourceCardId,
collectorNumber: doc.metafields.collector_number,
rarity,
finish: variant ? variant.finish : null
};
}
async function buildPreview({ shop, game, csvText, filename, uploadedBy }) {
const deps = getDeps();
const headerLine = String(csvText || '').split(/\r?\n/, 1)[0] || '';
assertPricingExportHeader(headerLine);
const rows = await parseCsvRows(csvText);
if (rows.length > MAX_IMPORT_ROWS) {
throw new TcgImportError(
`That file has more than ${MAX_IMPORT_ROWS.toLocaleString()} rows. Split it and upload in parts.`,
'TOO_MANY_ROWS'
);
}
// Classify first so set resolution only pays for rows that can match.
const classified = rows.map((row) => ({ row, fields: indexRowByLowercaseKey(row), ...classifyRow(row) }));
const setNames = [...new Set(
classified.filter((c) => c.status === 'single' || c.status === 'sealed')
.map((c) => String(c.fields.get('set name') || '').trim())
.filter(Boolean)
)];
const setMapRaw = setNames.length ? await deps.resolveSetCodes(setNames) : new Map();
const importDoc = await deps.TcgImport.create({
shop,
game,
status: 'preview',
filename,
uploadedAt: new Date(),
uploadedBy,
setMap: setNames.map((name) => ({
tcgSetName: name,
groupId: setMapRaw.get(name) ? setMapRaw.get(name).groupId : null,
setCode: setMapRaw.get(name) ? setMapRaw.get(name).setCode : null
})),
progress: { phase: 'preview', setsDone: [] }
});
const counts = {
totalRows: rows.length, zeroQty: 0, matched: 0, unmatched: 0, sealed: 0,
distinctProducts: 0, distinctSets: 0, units: 0
};
const distinctProducts = new Set();
const distinctSets = new Set();
const unmatchedSample = [];
let batch = [];
const flush = async () => {
if (!batch.length) return;
await deps.TcgImportLine.insertMany(batch);
batch = [];
};
for (const c of classified) {
const setName = String(c.fields.get('set name') || '').trim();
const productName = String(c.fields.get('product name') || '').trim();
const number = String(c.fields.get('number') || '').trim();
const resolved = setMapRaw.get(setName) || null;
let match;
if (c.status === 'zero_qty') {
counts.zeroQty++;
match = { status: 'zero_qty', reason: c.reason, setCode: null, cardUuid: null, collectorNumber: null, rarity: null, finish: null };
} else if (c.status === 'unsupported_line') {
counts.unmatched++;
match = { status: 'unsupported_line', reason: c.reason, setCode: null, cardUuid: null, collectorNumber: null, rarity: null, finish: null };
} else if (c.status === 'sealed') {
counts.sealed++;
// Sealed matching lands in PR 4; for now the row is recorded and
// reported as sealed so the merchant knows it was seen.
match = {
status: 'sealed',
reason: 'Sealed product — supported in a later release',
setCode: resolved ? resolved.setCode : null,
cardUuid: null, collectorNumber: null, rarity: null, finish: null
};
} else {
match = await matchLine({
setCode: resolved ? resolved.setCode : null,
setName, productName, number, finish: c.parsed.finish
}, game, deps);
if (match.status === 'matched') {
counts.matched++;
counts.units += c.parsed.quantity;
distinctProducts.add(`${match.setCode}|${match.cardUuid}`);
distinctSets.add(match.setCode);
} else {
counts.unmatched++;
}
}
if (match.status === 'unmatched' && unmatchedSample.length < UNMATCHED_SAMPLE_SIZE) {
unmatchedSample.push({ setName, productName, number, reason: match.reason });
}
batch.push({
importId: importDoc._id,
shop,
raw: {
tcgId: String(c.fields.get('tcgplayer id') || ''),
productLine: String(c.fields.get('product line') || ''),
setName,
productName,
number,
rarity: String(c.fields.get('rarity') || ''),
condition: String(c.fields.get('condition') || ''),
totalQuantity: parseInt(c.fields.get('total quantity'), 10) || 0,
marketplacePrice: toNumber(c.fields.get('tcg marketplace price')),
marketPrice: toNumber(c.fields.get('tcg market price'))
},
parsed: c.parsed || { finish: null, condition: null, quantity: 0 },
match,
result: { productId: null, variantId: null, action: null, error: null }
});
if (batch.length >= INSERT_BATCH_SIZE) await flush();
}
await flush();
counts.distinctProducts = distinctProducts.size;
counts.distinctSets = distinctSets.size;
await deps.TcgImport.findByIdAndUpdate(importDoc._id, { $set: { counts } });
return {
importId: importDoc._id,
counts,
setMap: importDoc.setMap,
unmatchedSample
};
}
module.exports = { buildPreview, matchLine, _setDeps, _resetDeps };- [ ] Step 5: Run the tests to verify they pass
Run: npx vitest run server/services/tcgImportService.test.js Expected: PASS, all cases.
- [ ] Step 6: Lint and commit
bash
npm run lint
git add server/services/tcgImportService.js server/services/tcgImportService.test.js server/services/__fixtures__/tcgplayer-pricing-export-sample.csv
git commit -m "Match an uploaded TCGplayer export against our card catalog"Task 5: The preview endpoint
Files:
- Create:
server/schemas/tcgImport.js - Create:
server/routes/tcgImport.js - Create:
server/routes/tcgImport.test.js - Modify:
server/schemas/index.js(add the barrel export) - Modify:
server/routes/api.js(mount the router beside the others, around line 642)
Interfaces:
Consumes:
buildPreview(Task 4);TcgImport,TcgImportLine(Task 3).Produces:
POST /api/tcg-import/preview— rawtext/csvbody, returns{ importId, counts, setMap, unmatchedSample }GET /api/tcg-import/:id— returns the import docGET /api/tcg-import/:id/lines?status=&page=&limit=— paginated linestcgImportIdParamSchema,tcgImportLinesQuerySchemainserver/schemas/tcgImport.js
[ ] Step 1: Write the failing tests
Create server/routes/tcgImport.test.js:
javascript
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
import express from 'express';
import request from 'supertest';
// The route module reads these off the request; dualModeAuth is stubbed by
// mounting our own middleware ahead of the router.
function appWith(router, overrides = {}) {
const app = express();
app.use((req, _res, next) => {
req.shop = 'test-store.myshopify.com';
req.accessToken = 'shpat_test';
req.user = { email: 'ufkesba@gmail.com' };
Object.assign(req, overrides);
next();
});
app.use('/api', router);
return app;
}
const HEADER = 'TCGplayer Id,Product Line,Set Name,Product Name,Title,Number,Rarity,Condition,TCG Market Price,TCG Direct Low,TCG Low Price With Shipping,TCG Low Price,Total Quantity,Add to Quantity,TCG Marketplace Price,My Store Reserve Quantity,My Store Price,Photo URL';
const ONE_ROW = '"376073","Magic","10th Edition","Adarkar Wastes","","347","R","Lightly Played","4.63","","5.3800","3.8900","2","0","5.3700","","",""';
let routerModule;
beforeEach(async () => {
routerModule = await import('./tcgImport.js');
routerModule._setDeps({
buildPreview: async ({ shop, game, csvText, filename }) => ({
importId: 'imp-1',
counts: { totalRows: 1, zeroQty: 0, matched: 1, unmatched: 0, sealed: 0, distinctProducts: 1, distinctSets: 1, units: 2 },
setMap: [{ tcgSetName: '10th Edition', groupId: 1, setCode: '10E' }],
unmatchedSample: [],
_echo: { shop, game, filename, bytes: csvText.length }
}),
TcgImport: { findOne: async () => ({ _id: 'imp-1', shop: 'test-store.myshopify.com', counts: {} }) },
TcgImportLine: {
find: () => ({ sort: () => ({ skip: () => ({ limit: () => ({ lean: async () => [{ _id: 'l1' }] }) }) }) }),
countDocuments: async () => 1
}
});
});
afterEach(() => routerModule._resetDeps());
describe('POST /api/tcg-import/preview', () => {
it('accepts a text/csv body and returns the report', async () => {
const res = await request(appWith(routerModule.default))
.post('/api/tcg-import/preview?game=mtg')
.set('Content-Type', 'text/csv')
.send(`${HEADER}\n${ONE_ROW}\n`);
expect(res.status).toBe(200);
expect(res.body.counts.matched).toBe(1);
});
it('rejects a request with no game rather than defaulting to mtg', async () => {
const res = await request(appWith(routerModule.default))
.post('/api/tcg-import/preview')
.set('Content-Type', 'text/csv')
.send(`${HEADER}\n${ONE_ROW}\n`);
expect(res.status).toBe(400);
expect(res.body.error).toMatch(/game/i);
});
it('rejects an unsupported game', async () => {
const res = await request(appWith(routerModule.default))
.post('/api/tcg-import/preview?game=chess')
.set('Content-Type', 'text/csv')
.send(`${HEADER}\n${ONE_ROW}\n`);
expect(res.status).toBe(400);
});
it('rejects an empty body', async () => {
const res = await request(appWith(routerModule.default))
.post('/api/tcg-import/preview?game=mtg')
.set('Content-Type', 'text/csv')
.send('');
expect(res.status).toBe(400);
expect(res.body.error).toMatch(/empty/i);
});
it('turns a TcgImportError into a 400 carrying its message, not a 500', async () => {
routerModule._setDeps({
buildPreview: async () => {
const e = new Error('That file is not a TCGplayer Pricing Custom Export.');
e.name = 'TcgImportError';
e.code = 'WRONG_TCGPLAYER_EXPORT';
throw e;
}
});
const res = await request(appWith(routerModule.default))
.post('/api/tcg-import/preview?game=mtg')
.set('Content-Type', 'text/csv')
.send(`${HEADER}\n${ONE_ROW}\n`);
expect(res.status).toBe(400);
expect(res.body.code).toBe('WRONG_TCGPLAYER_EXPORT');
expect(res.body.error).toMatch(/Pricing Custom Export/);
});
});
describe('GET /api/tcg-import/:id/lines', () => {
it('rejects a status outside the declared set', async () => {
const res = await request(appWith(routerModule.default))
.get('/api/tcg-import/507f1f77bcf86cd799439011/lines?status=maybe');
expect(res.status).toBe(400);
});
it('returns a page of lines for a valid status', async () => {
const res = await request(appWith(routerModule.default))
.get('/api/tcg-import/507f1f77bcf86cd799439011/lines?status=unmatched');
expect(res.status).toBe(200);
expect(res.body.lines).toHaveLength(1);
expect(res.body.total).toBe(1);
});
});- [ ] Step 2: Run the tests to verify they fail
Run: npx vitest run server/routes/tcgImport.test.js Expected: FAIL — Failed to resolve import "./tcgImport.js"
- [ ] Step 3: Write the Zod schemas
Create server/schemas/tcgImport.js:
javascript
/**
* TCGplayer import schemas.
*
* The CSV body itself is not Zod-validated — it is raw text handled by
* express.text and parsed by tcgImportService, which rejects a wrong export
* by header. These schemas cover the parameters around it.
*/
'use strict';
const { z } = require('zod');
// game is required with no default: a missing game is a bug to surface,
// not a value to fill in (CLAUDE.md §5.5).
const tcgImportPreviewQuerySchema = z.object({
game: z.string().min(1, 'game is required')
});
const tcgImportIdParamSchema = z.object({
id: z.string().regex(/^[a-f0-9]{24}$/i, 'id must be a Mongo ObjectId')
});
const tcgImportLinesQuerySchema = z.object({
status: z.enum(['matched', 'unmatched', 'zero_qty', 'unsupported_line', 'sealed']).optional(),
page: z.coerce.number().int().min(1).default(1),
limit: z.coerce.number().int().min(1).max(200).default(50)
});
const tcgImportConfirmSchema = z.object({
priceMode: z.enum(['sync', 'locked'])
});
module.exports = {
tcgImportPreviewQuerySchema,
tcgImportIdParamSchema,
tcgImportLinesQuerySchema,
tcgImportConfirmSchema
};In server/schemas/index.js, add beside the other spreads:
javascript
// TCGplayer import
...require('./tcgImport'),- [ ] Step 4: Write the route module
Create server/routes/tcgImport.js:
javascript
/**
* TCGplayer import routes.
*
* Mounted inside routes/api.js AFTER dualModeAuth and resolveTier, so every
* handler here already has req.shop, req.accessToken and req.tier.
*/
'use strict';
const express = require('express');
const router = express.Router();
const { validate } = require('../middleware/validate');
const {
tcgImportPreviewQuerySchema,
tcgImportIdParamSchema,
tcgImportLinesQuerySchema
} = require('../schemas');
const { isGameSupported } = require('../plugins');
const defaultService = require('../services/tcgImportService');
const TcgImport = require('../models/TcgImport');
const TcgImportLine = require('../models/TcgImportLine');
const logger = require('../utils/logger');
// The global express.json() at server/index.js:97 uses the 100kb default, and
// there is no multipart handler in this repo. A 1.3MB export would 413 before
// reaching any handler, so the upload route mounts its own text parser. Scoped
// to this one route on purpose: raising the global limit would widen the
// request surface of every other endpoint.
const csvBodyParser = express.text({ limit: '20mb', type: ['text/csv', 'text/plain'] });
let _deps = null;
function _setDeps(deps) { _deps = deps; }
function _resetDeps() { _deps = null; }
function getDeps() {
if (_deps) return _deps;
return { buildPreview: defaultService.buildPreview, TcgImport, TcgImportLine };
}
router.post(
'/tcg-import/preview',
csvBodyParser,
validate(tcgImportPreviewQuerySchema, { source: 'query' }),
async (req, res) => {
const { game } = req.query;
if (!isGameSupported(game)) {
return res.status(400).json({ error: `Game '${game}' is not supported.` });
}
const csvText = typeof req.body === 'string' ? req.body : '';
if (!csvText.trim()) {
return res.status(400).json({ error: 'The uploaded file was empty.' });
}
try {
const report = await getDeps().buildPreview({
shop: req.shop,
game,
csvText,
filename: req.get('X-Filename') || 'tcgplayer-export.csv',
uploadedBy: req.user ? req.user.email : undefined
});
return res.json(report);
} catch (error) {
if (error.name === 'TcgImportError') {
return res.status(400).json({ error: error.message, code: error.code });
}
logger.error('TCGplayer import preview failed', { shop: req.shop, error: error.message });
return res.status(500).json({ error: 'We could not read that file. Try the upload again.' });
}
}
);
router.get(
'/tcg-import/:id',
validate(tcgImportIdParamSchema, { source: 'params' }),
async (req, res) => {
// Scoped by shop: an import id from another store must 404, not leak.
const doc = await getDeps().TcgImport.findOne({ _id: req.params.id, shop: req.shop });
if (!doc) return res.status(404).json({ error: 'Import not found' });
return res.json(doc);
}
);
router.get(
'/tcg-import/:id/lines',
validate(tcgImportIdParamSchema, { source: 'params' }),
validate(tcgImportLinesQuerySchema, { source: 'query' }),
async (req, res) => {
const { status, page, limit } = req.query;
const deps = getDeps();
const query = { importId: req.params.id, shop: req.shop };
if (status) query['match.status'] = status;
const [lines, total] = await Promise.all([
deps.TcgImportLine.find(query)
.sort({ _id: 1 })
.skip((page - 1) * limit)
.limit(limit)
.lean(),
deps.TcgImportLine.countDocuments(query)
]);
return res.json({ lines, total, page, limit });
}
);
module.exports = router;
module.exports._setDeps = _setDeps;
module.exports._resetDeps = _resetDeps;In server/routes/api.js, beside the other router.use calls (around line 642):
javascript
router.use(require('./tcgImport'));- [ ] Step 5: Run the tests to verify they pass
Run: npx vitest run server/routes/tcgImport.test.js Expected: PASS, all cases.
If supertest is not already a devDependency, install it first: npm i -D supertest. Check package.json before adding — several route tests in this repo already use it.
- [ ] Step 6: Run the whole server suite
Run: npm test Expected: PASS. The barrel change in server/schemas/index.js is covered by server/schemas/schemas.test.js; if that suite fails, the export name collided with an existing one — rename ours, don't weaken the test.
- [ ] Step 7: Lint and commit
bash
npm run lint
git add server/schemas/tcgImport.js server/schemas/index.js server/routes/tcgImport.js server/routes/tcgImport.test.js server/routes/api.js
git commit -m "Let merchants upload a TCGplayer export and get back a match report"Task 6: Tier verdict and the price comparison sample
Everything so far reports what we matched. This task adds the two things that let a merchant decide: what this will cost them, and how our prices compare to theirs.
Files:
- Modify:
server/services/tcgImportService.js - Modify:
server/services/tcgImportService.test.js
Interfaces:
- Consumes:
planService.checkProductLimit(server/services/planService.js:142);priceLookupService.getPricesForCardandcalculateConditionPrice;pricingConfigService.getGamePricingConfig;PRICING_PLANSandTIER_KEYSfromserver/config/pricingPlans.js(both are exported — verified). - Produces:
buildPreviewnow also returns{ tierCheck, priceComparison }and persiststierCheckon the import doc.tierCheckis{ tier, limit, current, estimated, allowed, requiredTier, exact }.priceComparisonis an array of{ productName, setCode, condition, theirPrice, ourPrice }, at most 50 entries. _setDepsgains{ checkProductLimit, getPricesForCard, calculateConditionPrice, getGamePricingConfig }.
Verify before writing the implementation: getPricesForCard(cardUUID, rarity, finishTypes, config) defaults finishTypes to ['Normal', 'Foil'] (capitalized), but callers in this repo pass plugin-vocabulary finishes — buylistQuoteService.resolveBasisPrice passes [variant.finish] and then reads prices._rawPrices?.[finish]. Open server/services/priceLookupService.js:750 and confirm which key the returned object uses for a lowercase finish before writing the lookup below. If it turns out to be _rawPrices[finish] rather than prices[finish], use that — the tests here stub the function, so they will pass either way, and a wrong key would silently produce an empty comparison table (§5.4).
- [ ] Step 1: Write the failing tests
Append to server/services/tcgImportService.test.js:
javascript
describe('tier verdict', () => {
it('reports the conservative estimate and marks it inexact', async () => {
_setDeps(stubs({
checkProductLimit: async (_shop, _token, additional) => ({
allowed: 0 + additional <= 500, tier: 'free', limit: 500, current: 0
})
}));
const { tierCheck } = await buildPreview({
shop: 's.myshopify.com', game: 'mtg', csvText: CSV, filename: 'sample.csv', accessToken: 't'
});
expect(tierCheck.estimated).toBe(3);
expect(tierCheck.exact).toBe(false);
});
it('names the cheapest tier that would fit when the limit is exceeded', async () => {
_setDeps(stubs({
checkProductLimit: async () => ({ allowed: false, tier: 'free', limit: 500, current: 499 })
}));
const { tierCheck } = await buildPreview({
shop: 's.myshopify.com', game: 'mtg', csvText: CSV, filename: 'sample.csv', accessToken: 't'
});
expect(tierCheck.allowed).toBe(false);
// 499 current + 3 new = 502, past Free's 500 but inside Starter's 2,000.
expect(tierCheck.requiredTier).toBe('starter');
});
it('leaves requiredTier null when the current plan already fits', async () => {
_setDeps(stubs({
checkProductLimit: async () => ({ allowed: true, tier: 'rampUp', limit: 15000, current: 100 })
}));
const { tierCheck } = await buildPreview({
shop: 's.myshopify.com', game: 'mtg', csvText: CSV, filename: 'sample.csv', accessToken: 't'
});
expect(tierCheck.allowed).toBe(true);
expect(tierCheck.requiredTier).toBeNull();
});
it('persists tierCheck on the import document', async () => {
_setDeps(stubs({ checkProductLimit: async () => ({ allowed: true, tier: 'free', limit: 500, current: 0 }) }));
await buildPreview({ shop: 's.myshopify.com', game: 'mtg', csvText: CSV, filename: 'sample.csv', accessToken: 't' });
expect(saved.imports[0].tierCheck.tier).toBe('free');
});
});
describe('price comparison', () => {
it('pairs their marketplace price against our computed price', async () => {
_setDeps(stubs({
checkProductLimit: async () => ({ allowed: true, tier: 'free', limit: 500, current: 0 }),
getPricesForCard: async () => ({ nonfoil: 6.5, foil: 9.0 })
}));
const { priceComparison } = await buildPreview({
shop: 's.myshopify.com', game: 'mtg', csvText: CSV, filename: 'sample.csv', accessToken: 't'
});
const adarkar = priceComparison.find((p) => p.productName === 'Adarkar Wastes');
expect(adarkar.theirPrice).toBe(5.37);
expect(adarkar.condition).toBe('lp');
});
it('compares condition to condition, not their LP price to our NM price', async () => {
// The stub multiplier below is lp: 0.5, so a 6.50 NM price must be
// reported as 3.25 for a Lightly Played row. Without the condition
// step this test reports 6.50 and the comparison overstates our
// prices on every non-NM row — 2,855 of 2,944 in the real export.
_setDeps(stubs({
checkProductLimit: async () => ({ allowed: true, tier: 'free', limit: 500, current: 0 }),
getPricesForCard: async () => ({ nonfoil: 6.5 }),
getGamePricingConfig: () => ({
conditionVariants: { conditionMultipliers: { nm: 1.0, lp: 0.5, mp: 0.4, hp: 0.3, damaged: 0.2 } }
})
}));
const { priceComparison } = await buildPreview({
shop: 's.myshopify.com', game: 'mtg', csvText: CSV, filename: 'sample.csv', accessToken: 't'
});
const adarkar = priceComparison.find((p) => p.productName === 'Adarkar Wastes');
expect(adarkar.ourPrice).toBe(3.25);
});
it('samples at most 50 rows, however large the file', async () => {
_setDeps(stubs({
checkProductLimit: async () => ({ allowed: true, tier: 'free', limit: 500, current: 0 }),
getPricesForCard: async () => ({ nonfoil: 1 })
}));
const { priceComparison } = await buildPreview({
shop: 's.myshopify.com', game: 'mtg', csvText: CSV, filename: 'sample.csv', accessToken: 't'
});
expect(priceComparison.length).toBeLessThanOrEqual(50);
});
it('only ever samples matched rows', async () => {
_setDeps(stubs({
checkProductLimit: async () => ({ allowed: true, tier: 'free', limit: 500, current: 0 }),
getPricesForCard: async () => ({ nonfoil: 1 })
}));
const { priceComparison } = await buildPreview({
shop: 's.myshopify.com', game: 'mtg', csvText: CSV, filename: 'sample.csv', accessToken: 't'
});
expect(priceComparison.some((p) => p.productName === 'Charizard')).toBe(false);
expect(priceComparison.some((p) => p.productName === 'Forest')).toBe(false);
});
it('skips a card we have no price for rather than reporting zero', async () => {
_setDeps(stubs({
checkProductLimit: async () => ({ allowed: true, tier: 'free', limit: 500, current: 0 }),
getPricesForCard: async () => ({})
}));
const { priceComparison } = await buildPreview({
shop: 's.myshopify.com', game: 'mtg', csvText: CSV, filename: 'sample.csv', accessToken: 't'
});
expect(priceComparison).toEqual([]);
});
});Also extend the stubs() helper at the top of the file — add these two keys to the returned object, before ...overrides:
javascript
checkProductLimit: async () => ({ allowed: true, tier: 'free', limit: 500, current: 0 }),
getPricesForCard: async () => ({ nonfoil: 1, foil: 1 }),
calculateConditionPrice: (nmPrice, condition, multipliers) =>
Math.round(nmPrice * ((multipliers && multipliers[condition]) ?? 1) * 100) / 100,
// Synchronous, and takes the Store document (pricingConfigService.js:114).
getGamePricingConfig: () => ({
conditionVariants: { conditionMultipliers: { nm: 1.0, lp: 1.0, mp: 1.0, hp: 1.0, damaged: 1.0 } }
}),
Store: { findOne: () => ({ lean: async () => ({ shop: 's.myshopify.com' }) }) },- [ ] Step 2: Run the tests to verify they fail
Run: npx vitest run server/services/tcgImportService.test.js Expected: FAIL — tierCheck and priceComparison are undefined on the returned report.
- [ ] Step 3: Extend the implementation
In server/services/tcgImportService.js, add to the requires at the top:
javascript
const defaultPlanService = require('./planService');
const defaultPriceLookupService = require('./priceLookupService');
const defaultPricingConfigService = require('./pricingConfigService');
const Store = require('../models/Store');
const { PRICING_PLANS, TIER_KEYS } = require('../config/pricingPlans');Add to getDeps()'s returned object:
javascript
checkProductLimit: defaultPlanService.checkProductLimit,
getPricesForCard: defaultPriceLookupService.getPricesForCard,
calculateConditionPrice: defaultPriceLookupService.calculateConditionPrice,
getGamePricingConfig: defaultPricingConfigService.getGamePricingConfig,
Store,Add these two constants beside UNMATCHED_SAMPLE_SIZE:
javascript
// Sampled, not exhaustive: pricing every matched card means preloading price
// snapshots for hundreds of sets inside an HTTP request. §5.6 requires those
// reads to stream, and even streaming that is minutes. The UI labels this a
// sample and claims no store-wide average — an average we did not compute
// would be a fabricated number.
const PRICE_SAMPLE_SIZE = 50;
const PRICE_SAMPLE_TOP_BY_VALUE = 25;Add these two helpers above buildPreview:
javascript
/**
* The cheapest tier whose productLimit clears `current + estimated`.
* Enterprise is excluded: it has no Managed Pricing plan and is only ever
* reached through a manual tierOverride (see pricingPlans.js).
*/
function cheapestTierThatFits(needed) {
for (const key of TIER_KEYS) {
if (key === 'enterprise') continue;
const plan = PRICING_PLANS[key]; // eslint-disable-line security/detect-object-injection -- key comes from the hardcoded TIER_KEYS array
if (plan.productLimit >= needed) return key;
}
return 'enterprise';
}
/**
* 25 highest-value rows plus 25 spread evenly through the rest, so the table
* shows both the cards that matter and a representative middle.
*/
function choosePriceSample(matchedLines) {
const byValue = [...matchedLines].sort(
(a, b) => (b.raw.marketplacePrice || 0) - (a.raw.marketplacePrice || 0)
);
const top = byValue.slice(0, PRICE_SAMPLE_TOP_BY_VALUE);
const rest = byValue.slice(PRICE_SAMPLE_TOP_BY_VALUE);
const wanted = PRICE_SAMPLE_SIZE - top.length;
if (rest.length <= wanted) return [...top, ...rest];
const stride = Math.floor(rest.length / wanted);
const spread = [];
for (let i = 0; i < wanted; i++) spread.push(rest[i * stride]);
return [...top, ...spread];
}Inside buildPreview, collect matched lines as you build each batch — add const matchedLines = []; beside const unmatchedSample = [];, and inside the match.status === 'matched' branch push the line document you are about to batch (move the batch.push({...}) object into a const lineDoc = {...} first, then batch.push(lineDoc) and if (match.status === 'matched') matchedLines.push(lineDoc);).
Then replace the return { importId, counts, setMap, unmatchedSample } block at the end with:
javascript
counts.distinctProducts = distinctProducts.size;
counts.distinctSets = distinctSets.size;
const limitResult = await deps.checkProductLimit(shop, accessToken, counts.distinctProducts);
const needed = (limitResult.current || 0) + counts.distinctProducts;
const tierCheck = {
tier: limitResult.tier,
limit: limitResult.limit,
current: limitResult.current,
estimated: counts.distinctProducts,
allowed: limitResult.allowed,
// Conservative: cards the store already has are counted again, because
// knowing otherwise needs the managed-collection cache. The merchant
// can ask for an exact count, which sets exact = true (PR 2).
requiredTier: limitResult.allowed ? null : cheapestTierThatFits(needed),
exact: false
};
// The merchant's price is per condition ("Lightly Played", $5.37), so ours
// must be too. getPricesForCard returns the NM-level price per finish;
// without the condition step below, every non-NM row would compare their
// LP price against our NM price and overstate us — 2,855 of the reference
// export's 2,944 stocked rows are non-NM. Same two-step recipe as
// shopifyAPI.ensureVariantForConditionAndInventory (shopifyAPI.js:1556).
// getGamePricingConfig is SYNCHRONOUS and takes the Store *document*, not a
// shop string (pricingConfigService.js:114) — verified, not assumed.
const storeDoc = await deps.Store.findOne({ shop }).lean();
const gameConfig = deps.getGamePricingConfig(storeDoc, game);
const conditionMultipliers = gameConfig
&& gameConfig.conditionVariants
&& gameConfig.conditionVariants.conditionMultipliers;
const priceComparison = [];
for (const line of choosePriceSample(matchedLines)) {
const prices = await deps.getPricesForCard(
line.match.cardUuid, line.match.rarity, [line.match.finish], gameConfig
);
const nmPrice = prices ? prices[line.match.finish] : null;
if (typeof nmPrice !== 'number' || nmPrice <= 0) continue;
const ourPrice = deps.calculateConditionPrice(
nmPrice, line.parsed.condition, conditionMultipliers
);
priceComparison.push({
productName: line.raw.productName,
setCode: line.match.setCode,
condition: line.parsed.condition,
theirPrice: line.raw.marketplacePrice,
ourPrice
});
}
await deps.TcgImport.findByIdAndUpdate(importDoc._id, { $set: { counts, tierCheck } });
return { importId: importDoc._id, counts, setMap: importDoc.setMap, unmatchedSample, tierCheck, priceComparison };Add accessToken to buildPreview's destructured parameter list: async function buildPreview({ shop, game, csvText, filename, uploadedBy, accessToken }).
In server/routes/tcgImport.js, pass it through — add accessToken: req.accessToken, to the buildPreview call.
- [ ] Step 4: Run the tests to verify they pass
Run: npx vitest run server/services/tcgImportService.test.js server/routes/tcgImport.test.js Expected: PASS, all cases.
- [ ] Step 5: Lint and commit
bash
npm run lint
git add server/services/tcgImportService.js server/services/tcgImportService.test.js server/routes/tcgImport.js
git commit -m "Tell merchants what a TCGplayer import will cost and how our prices compare"Task 7: The import page
Files:
- Create:
client/src/pages/catalog/CatalogImportPage.jsx - Create:
client/src/pages/catalog/CatalogImportPage.test.jsx - Modify:
client/src/utils/api.js(add the three calls) - Modify:
client/src/App.jsx(add the route inside the/catalog/:gamelayout, besidesetsat line 142)
Interfaces:
Consumes:
POST /api/tcg-import/preview,GET /api/tcg-import/:id/lines(Task 5).Produces: route
/catalog/:game/import.[ ] Step 1: Add the API calls
In client/src/utils/api.js, beside the other exported helpers:
javascript
/**
* Upload a TCGplayer Pricing Custom Export for analysis.
* Sent as a raw text/csv body: there is no multipart handler on the server,
* and the global express.json() limit (100kb) is far below a real export.
* @param {string} game - never defaulted; comes from the :game route param
* @param {string} csvText - file contents, read client-side via FileReader
* @param {string} filename
*/
export async function previewTcgImport(game, csvText, filename) {
const res = await api.post('/tcg-import/preview', csvText, {
params: { game },
headers: { 'Content-Type': 'text/csv', 'X-Filename': filename },
});
return res.data;
}
export async function getTcgImport(id) {
const res = await api.get(`/tcg-import/${id}`);
return res.data;
}
export async function getTcgImportLines(id, { status, page = 1, limit = 50 } = {}) {
const res = await api.get(`/tcg-import/${id}/lines`, { params: { status, page, limit } });
return res.data;
}- [ ] Step 2: Write the failing component test
Create client/src/pages/catalog/CatalogImportPage.test.jsx:
jsx
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { render, screen, waitFor } from '@testing-library/react';
import userEvent from '@testing-library/user-event';
import { MemoryRouter, Routes, Route } from 'react-router-dom';
import CatalogImportPage from './CatalogImportPage.jsx';
import * as api from '../../utils/api.js';
function renderAt(game = 'mtg') {
return render(
<MemoryRouter initialEntries={[`/catalog/${game}/import`]}>
<Routes>
<Route path="/catalog/:game/import" element={<CatalogImportPage />} />
</Routes>
</MemoryRouter>
);
}
const REPORT = {
importId: 'imp-1',
counts: { totalRows: 8514, zeroQty: 5570, matched: 2920, unmatched: 24, sealed: 4, distinctProducts: 2829, distinctSets: 319, units: 4082 },
setMap: [],
unmatchedSample: [{ setName: 'Art Series: Lorwyn Eclipsed', productName: 'Ancient Grudge', number: '5', reason: "We don't carry Art Series: Lorwyn Eclipsed yet" }],
tierCheck: { tier: 'free', limit: 500, current: 0, estimated: 2829, allowed: false, requiredTier: 'rampUp', exact: false },
priceComparison: [{ productName: 'Adarkar Wastes', setCode: '10E', condition: 'lp', theirPrice: 5.37, ourPrice: 6.5 }],
};
beforeEach(() => vi.restoreAllMocks());
describe('CatalogImportPage', () => {
it('sends the game from the route param, never a default', async () => {
const spy = vi.spyOn(api, 'previewTcgImport').mockResolvedValue(REPORT);
renderAt('mtg');
const input = screen.getByLabelText(/tcgplayer export/i);
await userEvent.upload(input, new File(['TCGplayer Id,Total Quantity,TCG Marketplace Price\n'], 'export.csv', { type: 'text/csv' }));
await waitFor(() => expect(spy).toHaveBeenCalled());
expect(spy.mock.calls[0][0]).toBe('mtg');
});
it('shows every bucket count after a successful preview', async () => {
vi.spyOn(api, 'previewTcgImport').mockResolvedValue(REPORT);
renderAt();
await userEvent.upload(screen.getByLabelText(/tcgplayer export/i), new File(['x'], 'export.csv', { type: 'text/csv' }));
await screen.findByText('2,829');
expect(screen.getByText(/2,920/)).toBeInTheDocument();
expect(screen.getByText(/5,570/)).toBeInTheDocument();
});
it('explains that the estimate is conservative when the tier is exceeded', async () => {
vi.spyOn(api, 'previewTcgImport').mockResolvedValue(REPORT);
renderAt();
await userEvent.upload(screen.getByLabelText(/tcgplayer export/i), new File(['x'], 'export.csv', { type: 'text/csv' }));
await screen.findByText(/Growth/);
expect(screen.getByText(/cards you have already synced are not excluded yet/i)).toBeInTheDocument();
});
it('labels the price table a sample and claims no average', async () => {
vi.spyOn(api, 'previewTcgImport').mockResolvedValue(REPORT);
renderAt();
await userEvent.upload(screen.getByLabelText(/tcgplayer export/i), new File(['x'], 'export.csv', { type: 'text/csv' }));
await screen.findByText(/sample of/i);
expect(screen.queryByText(/on average/i)).not.toBeInTheDocument();
});
it('shows the server message when the wrong export is uploaded', async () => {
vi.spyOn(api, 'previewTcgImport').mockRejectedValue({
response: { status: 400, data: { error: 'That file is not a TCGplayer Pricing Custom Export.', code: 'WRONG_TCGPLAYER_EXPORT' } },
});
renderAt();
await userEvent.upload(screen.getByLabelText(/tcgplayer export/i), new File(['x'], 'wrong.csv', { type: 'text/csv' }));
await screen.findByText(/not a TCGplayer Pricing Custom Export/i);
});
it('rejects an oversized file client-side, naming the server limit', async () => {
const spy = vi.spyOn(api, 'previewTcgImport').mockResolvedValue(REPORT);
renderAt();
const big = new File(['x'], 'big.csv', { type: 'text/csv' });
Object.defineProperty(big, 'size', { value: 21 * 1024 * 1024 });
await userEvent.upload(screen.getByLabelText(/tcgplayer export/i), big);
await screen.findByText(/20MB/i);
expect(spy).not.toHaveBeenCalled();
});
});- [ ] Step 3: Run the test to verify it fails
Run: npx vitest run --config client/vitest.config.js client/src/pages/catalog/CatalogImportPage.test.jsx Expected: FAIL — module not found.
If that config path is wrong, find the client test command in package.json's test:client script and use it with the single-file path.
- [ ] Step 4: Write the component
Create client/src/pages/catalog/CatalogImportPage.jsx:
jsx
/* global FileReader */
import { useState } from 'react';
import { useParams } from 'react-router-dom';
import { Upload } from 'lucide-react';
import { Alert, Button, Card, Text } from '../../components/retroui';
import { previewTcgImport } from '../../utils/api';
// Mirrors the server cap in server/routes/tcgImport.js
// (express.text({ limit: '20mb' })). UX only — the server enforces.
const MAX_BYTES = 20 * 1024 * 1024;
const TIER_LABELS = {
free: 'Free', starter: 'Starter', rampUp: 'Growth',
large: 'Established', enterprise: 'Enterprise',
};
const n = (value) => (typeof value === 'number' ? value.toLocaleString() : '—');
const money = (value) => (typeof value === 'number' ? `$${value.toFixed(2)}` : '—');
export default function CatalogImportPage() {
// Never defaulted — a missing game is a bug to surface, not a value to fill in.
const { game } = useParams();
const [report, setReport] = useState(null);
const [error, setError] = useState(null);
const [busy, setBusy] = useState(false);
const handleFile = (event) => {
const file = event.target.files && event.target.files[0];
if (!file) return;
setError(null);
setReport(null);
if (file.size > MAX_BYTES) {
setError('That file is larger than 20MB. Split it and upload in parts.');
return;
}
setBusy(true);
const reader = new FileReader();
reader.onload = async () => {
try {
setReport(await previewTcgImport(game, String(reader.result), file.name));
} catch (err) {
setError(err?.response?.data?.error || 'We could not read that file. Try the upload again.');
} finally {
setBusy(false);
}
};
reader.onerror = () => { setError('We could not read that file.'); setBusy(false); };
reader.readAsText(file);
};
return (
<div className="space-y-6">
<div>
<Text as="h2" className="text-xl font-bold">Import from TCGplayer</Text>
<Text as="p" className="text-sm">
Upload a Pricing Custom Export from your TCGplayer Seller Portal. Nothing is created
in your store until you review this report and confirm.
</Text>
</div>
<div>
<label htmlFor="tcg-file" className="block text-sm font-medium mb-2">
TCGplayer export (.csv)
</label>
<input
id="tcg-file"
type="file"
accept=".csv,text/csv"
onChange={handleFile}
disabled={busy}
/>
</div>
{busy && <Text as="p" className="text-sm">Reading your file…</Text>}
{error && (
<Alert status="error">
<Alert.Description>{error}</Alert.Description>
</Alert>
)}
{report && (
<div className="space-y-6">
<Card className="p-4">
<Text as="h3" className="font-bold mb-2">What we found</Text>
<dl className="grid grid-cols-2 gap-2 text-sm sm:grid-cols-4">
<div><dt>Products to create</dt><dd className="font-bold">{n(report.counts.distinctProducts)}</dd></div>
<div><dt>Rows matched</dt><dd>{n(report.counts.matched)}</dd></div>
<div><dt>Rows we couldn't match</dt><dd>{n(report.counts.unmatched)}</dd></div>
<div><dt>Rows with no stock (skipped)</dt><dd>{n(report.counts.zeroQty)}</dd></div>
<div><dt>Units</dt><dd>{n(report.counts.units)}</dd></div>
<div><dt>Sets</dt><dd>{n(report.counts.distinctSets)}</dd></div>
<div><dt>Sealed rows</dt><dd>{n(report.counts.sealed)}</dd></div>
</dl>
</Card>
{report.tierCheck && !report.tierCheck.allowed && (
<Alert status="warning">
<Alert.Description>
<div className="space-y-2 text-sm">
<div>
This import could add up to {n(report.tierCheck.estimated)} products. Your{' '}
{TIER_LABELS[report.tierCheck.tier] || report.tierCheck.tier} plan allows{' '}
{n(report.tierCheck.limit)}, and you have {n(report.tierCheck.current)} today.
You would need the {TIER_LABELS[report.tierCheck.requiredTier] || report.tierCheck.requiredTier} plan.
</div>
{!report.tierCheck.exact && (
<div>
This is a high estimate — cards you have already synced are not excluded yet.
</div>
)}
<a href="/settings/billing" className="underline font-medium">View plans</a>
</div>
</Alert.Description>
</Alert>
)}
{report.priceComparison?.length > 0 && (
<Card className="p-4">
<Text as="h3" className="font-bold">How our pricing compares</Text>
<Text as="p" className="text-sm mb-2">
We price from market data and your condition multipliers, so prices move with the
market instead of staying where you set them. Sample of {report.priceComparison.length} of
your listings:
</Text>
<div className="overflow-x-auto">
<table className="w-full text-sm">
<thead>
<tr>
<th className="text-left">Card</th>
<th className="text-left">Set</th>
<th className="text-left">Condition</th>
<th className="text-right">Your price</th>
<th className="text-right">Ours</th>
</tr>
</thead>
<tbody>
{report.priceComparison.map((r, i) => (
<tr key={`${r.setCode}-${r.productName}-${r.condition}-${i}`}>
<td>{r.productName}</td>
<td>{r.setCode}</td>
<td className="uppercase">{r.condition}</td>
<td className="text-right">{money(r.theirPrice)}</td>
<td className="text-right">{money(r.ourPrice)}</td>
</tr>
))}
</tbody>
</table>
</div>
</Card>
)}
{report.unmatchedSample?.length > 0 && (
<Card className="p-4">
<Text as="h3" className="font-bold mb-2">
Rows we couldn't match ({n(report.counts.unmatched)})
</Text>
<ul className="text-sm space-y-1">
{report.unmatchedSample.map((r, i) => (
<li key={`${r.setName}-${r.productName}-${i}`}>
<span className="font-medium">{r.productName}</span>
{r.setName ? ` (${r.setName})` : ''} — {r.reason}
</li>
))}
</ul>
</Card>
)}
<Alert status="info">
<Alert.Description>
<Text as="span" className="text-sm">
Running the import is coming next. This report is saved — you can come back to it.
</Text>
</Alert.Description>
</Alert>
<Button disabled>
<Upload className="h-4 w-4 mr-1" />
Run import (coming soon)
</Button>
</div>
)}
</div>
);
}Before running the test, open client/src/components/retroui/index.js and confirm Alert, Button, Card, and Text are all exported from the barrel, and that Alert.Description and Alert accept a status prop — MarketplaceSettings.jsx:135 uses exactly this shape. If any import name differs, match the existing usage rather than inventing one.
- [ ] Step 5: Add the route
In client/src/App.jsx, inside the /catalog/:game layout route, beside <Route path="sets" ... /> at line 142:
jsx
<Route path="import" element={<CatalogImportPage />} />and add the import beside the other catalog page imports:
jsx
import CatalogImportPage from './pages/catalog/CatalogImportPage';- [ ] Step 6: Run the tests to verify they pass
Run: npm run test:client Expected: PASS.
- [ ] Step 7: Build and lint
bash
npm run build
npm run lintExpected: both exit 0.
- [ ] Step 8: Commit
bash
git add client/src/pages/catalog/CatalogImportPage.jsx client/src/pages/catalog/CatalogImportPage.test.jsx client/src/utils/api.js client/src/App.jsx
git commit -m "Show merchants what a TCGplayer export would bring into their store"Task 8: Verify against the real export, then document
The whole point of PR 1 is learning the true match rate before writing any products. This task is where that number gets measured.
Files:
Create:
docs/guides/tcgplayer-import.mdModify:
docs/.vitepress/config.js(add the sidebar entry)[ ] Step 1: Run the full check suite
bash
npm test
npm run test:client
npm run lint
npm run buildExpected: all four exit 0.
- [ ] Step 2: Read the per-file coverage table by hand
Run: npm run test:coverage
Two separate bars. The command's own gate is the repo-wide ratchet in vitest.config.js (thresholds: lines 51 / functions 59 / branches 44 / statements 51) — it is real and the run fails if the branch drops below it, so treat a non-zero exit as a genuine failure and never lower a threshold to pass. That gate is global, though, so it does not enforce the per-file bar. Read the per-file table yourself and confirm ≥70% on all four metrics for: server/utils/tcgplayerCsv.js, server/services/tcgcsvGroupService.js, server/services/tcgImportService.js, server/routes/tcgImport.js, server/models/TcgImport.js, server/models/TcgImportLine.js. If any file is short, add tests for the uncovered branch — do not lower the bar.
- [ ] Step 3: Measure the real match rate
With docker compose up -d running and MTG data loaded (from Task 2 Step 5), start the dev server and upload the real export at /catalog/mtg/import.
Record in the PR description: rows total, matched, unmatched, zero-quantity, distinct products, distinct sets, and the top five unmatched reasons by frequency.
Expected from the design's measurements: ~2,944 stocked rows, ~2,829 distinct products, ~319 sets, and roughly 24 unmatched from the 10 unresolvable sets. If the unmatched count is materially higher than ~25, stop and report it — that means the catalog-side match is weaker than the set-side bridge suggested, and PR 2 should not be planned until it is understood.
- [ ] Step 4: Write the merchant guide
Create docs/guides/tcgplayer-import.md:
markdown
# Import your TCGplayer inventory
Bring the singles you already have listed on TCGplayer into your Shopify store,
with their conditions and quantities.
## Export from TCGplayer
1. In the TCGplayer Seller Portal, open **Pricing**.
2. Export a **Pricing Custom Export**, making sure **Total Quantity** and
**TCG Marketplace Price** are among the selected columns.
3. Save the `.csv`.
The Collection export will not work — it has no quantity or price columns.
## Upload it
Go to **Catalog → MTG → Import** and choose your file. Nothing is created in
your store yet: you get a report first.
## What the report tells you
- **Products to create** — one per distinct card, however many conditions you
stock it in.
- **Rows matched** — lines we found in our catalog.
- **Rows we couldn't match** — with the reason for each. Art Series, Promo Packs
and playtest cards are the usual ones; we don't carry them yet.
- **Rows with no stock** — any line with `Total Quantity` of 0 is skipped.
- **How our pricing compares** — a sample of your listings beside the price we
would set. We price from market data and your condition multipliers, so your
prices move with the market instead of staying where you set them.
## Conditions
Your export carries the condition in each row (`Lightly Played`, `Near Mint Foil`,
and so on). We create only the conditions you actually stock, plus Near Mint —
we do not turn on condition variants for your whole catalog.
## Plan limits
If the import would take you past your plan's product limit, the report says so
and names the plan you would need. The estimate is deliberately high: cards you
have already synced are not excluded from it.Add to the guides list in docs/.vitepress/config.js, beside the other /guides/ entries:
javascript
{ text: 'TCGplayer Import', link: '/guides/tcgplayer-import' },- [ ] Step 5: Build the docs
Run: npm run docs:build Expected: exit 0.
- [ ] Step 6: Commit and open the PR
bash
git add docs/guides/tcgplayer-import.md docs/.vitepress/config.js
git commit -m "Explain how to bring a TCGplayer export into a Shopify store"
git push -u origin HEADThe PR description must contain:
- The measured numbers from Step 3.
- The §5.1 parity declaration: mtg — implemented; pokemon — exempt, no real export available so its
Product Lineliteral is unverified and inventing one is §5.4; riftbound — exempt, same reason. - A note that nothing in this PR writes to Shopify.
- The Task 2 Step 5 result: how many
mtg_setsdocuments carrytcgplayerGroupId.
Self-Review
Spec coverage. Every PR-1 requirement in the spec maps to a task: format module and the Unopened interception → Task 1; set bridge and its Redis cache → Task 2; both collections with §5.2 round-trip tests → Task 3; set-scoped matching, ambiguity handling, and rarity-from-catalog → Task 4; the route surface with Zod schemas and the route-scoped body parser → Task 5; tier verdict and the 50-row price sample → Task 6; upload UI → Task 7; verification and the merchant guide → Task 8.
Deliberately not in this plan, because they belong to later PRs: the cardIdentities filter on syncSetDirect, setInventoryQuantity, the queue and processor, the price_locked metafield, the exact-count route, the /confirm route, and sealed matching. tcgImportConfirmSchema is written in Task 5 but wired in PR 2 — it is three lines and keeps the schema file coherent.
Known gaps carried forward. Sealed rows are recorded and reported in PR 1 but not matched; the line's match.status is sealed with a plain reason. SealedProduct.identifiers.tcgplayerProductId remains unverified until PR 4, as the spec states.