Buylist URL + CSV Import Entry Points — Design

Issue: #402 — "Buylist: URL and CSV import buttons on Buy Order" Date: 2026-07-26 Status: Approved (Brent, 2026-07-26)

Problem

The First Pass review asked for two import entry points next to manual entry on the Buy Order tab: paste a list URL and ingest it, and upload a CSV of cards/quantities.

The issue points at docs/superpowers/plans/2026-07-18-buylist-import-connectors.md and says to follow it. That plan already shipped in PR #376 (merged 2026-07-19). What exists on main today:

• parseCsvRows, parseManaBoxCsv, parseTcgplayerCsv in server/services/buylistQuoteService.js
• format: z.enum(['plain','manabox','tcgplayer']) on createBuylistOrderSchema
• Dispatch in generateQuote
• A Format dropdown inside Paste List mode, plus an Upload file text link that only renders once the dropdown is changed (client/src/pages/buylist/NewBuyInTab.jsx)

So the parser half of "CSV import" is done. What remains is genuinely two things:

1. URL import — does not exist. No plan, no spec section, and no precedent anywhere in the server for user-supplied input reaching an outbound fetch.
2. CSV discoverability — the capability exists but is not an entry point. It is a dropdown nested inside another mode.

Vendor research (grounding)

Direction from Brent (2026-07-26): use official/sanctioned routes only; do not scrape, since scraped routes get blocked by bot protection.

Source	Read API	Sanctioned?	Decision
Archidekt	https://archidekt.com/api/decks/<id>/, unauthenticated, undocumented	Yes, explicitly by staff. Archidekt staff member Michael, forum thread 2832338: "You're more than welcome to use our API for whatever you want (just know that you might hit our rate limiter if you hit it too hard)." Docs are deliberately absent: "The main reason we don't provide API docs is because guaranteeing commitment to keeping them up to date with how frequently we make changes to Archidekt."	Build it
Moxfield	No public API. api2.moxfield.com is unofficial	No. Access requires a support-granted user-agent allowlist, and developers report Cloudflare still rejecting even allowlisted agents (moxfield-public#143). A direct fetch of moxfield.com during this research returned HTTP 403.	Registered but disabled — pending Brent emailing Moxfield support
ManaBox	No API. Collection "share" links are browser HTML pages	Would require scraping	Skip — its CSV export is already supported
TCGplayer	Official API exists, but new applications closed since late 2024 (post-eBay); no new keys issued	Cannot obtain access	Skip — its CSV export is already supported

Known tension, accepted knowingly

Archidekt staff verbally sanction API use, but their written Terms of Service §3 Acceptable Use prohibits users from "use software or automated agents or scripts to produce multiple accounts on the Site, or to generate automated searches, requests, or queries to the Site."

Read in context the clause targets abuse — it is bracketed by multi-account creation. One request per explicit merchant click, with an identifying User-Agent and respect for rate limiting, is materially different from automated searching. Brent was shown this tension and approved proceeding. Mitigations below (own-endpoint rate limit, single request per action, identifying UA) exist partly to keep us on the right side of it.

Scope

In: merchant-side Buy Order tab (authenticated) only. Out: the public customer portal. It uses entirely separate schemas (publicQuoteSchema, publicOrderCreateSchema in server/routes/publicBuylist.js) and is untouched by this design — unauthenticated traffic must never be able to trigger an outbound fetch. This is a deliberate boundary, not an oversight.

Architecture

New module server/services/deckUrlImportService.js holding a connector registry. It produces the existing ParsedLine shape and hands off; every downstream stage (matchCatalogLine, resolveBasisPrice, applyBuylistRate, computeOrderTotals) is untouched and stays plugin-driven.

NewBuyInTab (From URL)
  → POST /api/buylist/orders { customer, game, sourceUrl }
    → createBuylistOrderSchema (exactly-one-of rawText | lines | sourceUrl)
      → generateQuote({ store, game, sourceUrl })
        → deckUrlImportService.importDeckUrl(sourceUrl)
          → identify connector → fetch → map → ParsedLine[]
        → [unchanged] matchCatalogLine → resolveBasisPrice → applyBuylistRate

Connector shape

{
  id: 'archidekt',
  label: 'Archidekt',
  enabled: true,
  games: ['mtg'],
  hosts: ['archidekt.com', 'www.archidekt.com'],
  extractId(url),        // URL object → deck id string, or null
  buildApiUrl(deckId),   // deck id → the URL WE construct
  toParsedLines(json)    // response → ParsedLine[]
}

Moxfield is registered with enabled: false so a pasted Moxfield URL yields a specific, honest message rather than a generic "unsupported URL". Enabling it later is a flag flip plus toParsedLines — not a redesign.

Security model

The pasted URL is never fetched. It is parsed for an identifier; the request URL is constructed by us against a hardcoded host.

1. new URL(input) — malformed input rejected up front
2. Protocol must be https:
3. url.hostname must exactly equal an entry in the connector's hosts (no suffix/endsWith matching — archidekt.com.evil.com must fail)
4. Path must match ^/decks/(\d+) — the captured id is digits only
5. Fetch https://archidekt.com/api/decks/${deckId}/ — a string we built

This removes the SSRF surface rather than filtering it: no redirect chasing, no DNS rebinding window, no internal-IP reachability, no userinfo (https://archidekt.com@evil.com/) confusion, since only the parsed hostname is compared and only a numeric id survives into the constructed URL.

Additional controls:

• redirect: 'error' — never follow a redirect
• 10s timeout via the existing server/utils/abortController.js
• Response size ceiling before JSON.parse (a real 109-card deck was ~331 KB; cap at 5 MB)
• User-Agent identifying LGS Forge with a contact URL — good citizenship, and what Archidekt's staff implicitly asked for
• The POST /api/buylist/orders URL path is rate-limited via the existing server/middleware/rateLimiter.js, so a merchant account cannot be used to hammer Archidekt
• Upstream failures map to merchant-readable messages; HTTP 429 gets its own ("Archidekt is rate-limiting requests; try again shortly") rather than a generic failure

Field mapping

Every literal below was verified against a real response captured 2026-07-26 from https://archidekt.com/api/decks/365563/ (HTTP 200, 330907 bytes). Per CLAUDE.md §5.4, nothing here is assumed.

ParsedLine field	Archidekt path	Observed value
quantity	cards[].quantity	1
name	cards[].card.oracleCard.name	"Pact of Negation"
setCode	cards[].card.edition.editioncode, uppercased	"a25" → "A25"
collectorNumber	cards[].card.collectorNumber	"68"
condition	(absent from decklists)	'nm'
rawLine	composed	1x Pact of Negation [A25] 68

Two traps worth naming:

• collectorNumber lives on card, not on card.oracleCard (where name lives)
• editioncode is lowercase in the response and must be uppercased to match this repo's set-code convention

Finish is dropped. cards[].modifier carries "Normal" / "Foil" (confirmed by card.options: ["Normal","Foil"]). The existing parseCardList and both CSV connectors also accept no explicit finish — it is re-derived downstream from the matched catalog doc. Dropping it is parity, not regression. (Note the vocabulary is "Normal", not "nonfoil" — the exact §5.4 trap.)

Maybeboard exclusion

Archidekt decks carry cards the customer is not selling. The response has:

"categories": [ { "name": "Maybeboard", "includedInDeck": false, ... },
                { "name": "Sideboard",  "includedInDeck": true,  ... } ]

and each card has categories: ["Maybeboard"].

Build the excluded name set from deck.categories[] where includedInDeck === false, then drop any card whose categories intersects it. Do not hardcode the string "Maybeboard" — categories are user-defined per deck (the sample deck contained custom names like "Fetches", "Wincon", "Lifegain"), and a merchant with a renamed maybeboard would otherwise get quoted cards the customer never offered. Note "Sideboard" had includedInDeck: true in the sample, so it is correctly included.

Lines are capped at the existing MAX_LINES (500), same as the CSV connectors.

Game restriction

The connector declares games: ['mtg']; a sourceUrl submitted with any other game is rejected with a clear message. The response's top-level game was null in the sample, so it cannot be trusted to self-identify. Silently matching a Magic decklist against the Pokemon catalog is the §5.3 broken-join failure shape — every line would come back unmatched with no explanation.

CSV auto-detection

format gains 'auto' and becomes the client's default for uploads. detectCsvFormat(headerRow) keys on tokens unique to each layout, both already constants in buylistQuoteService.js:

• ManaBox → header contains ManaBox ID
• TCGplayer → header contains TCGplayer Id
• neither → typed error naming the supported formats, rather than a silent mismatch

Detection is server-side (CLAUDE.md rule 7: server enforces, client decorates). 'plain' | 'manabox' | 'tcgplayer' remain valid explicit values, so existing callers are unaffected.

Client

The mode row becomes four entry points:

[ Search & Select ] [ Paste List ] [ Upload CSV ] [ From URL ]

• Upload CSV — file input, sends format: 'auto', surfaces the detected format back to the merchant
• From URL — URL field, sends sourceUrl, help text naming Archidekt as supported
• The Format dropdown is removed; Paste List returns to a plain textarea

Failure modes reviewed (CLAUDE.md §5)

Mode	Applies?	Handling
§5.1 One-Game Fix	Partly	No plugin code, per-game model, or importer touched. Archidekt is MTG-only by the vendor's product, declared explicitly rather than assumed. mtg = supported; pokemon/riftbound = rejected with a clear message, not silently mismatched
§5.2 Silent Schema Drop	No	Nothing new persisted; BuylistOrder.lines[] shape unchanged
§5.3 Broken Join	Yes	Set code uppercased to match catalog convention; game restriction prevents cross-catalog matching
§5.4 Invented Vocabulary	Yes	Every field path and value cited to the captured response; "Normal" (not "nonfoil") noted explicitly
§5.5 Phantom Default	Yes	No identity defaults; game still flows from client → schema → service
§5.6 Sort Before Project	No	No new aggregations
§5.9 Dead Scaffold	Yes	The disabled Moxfield connector is consumed in the same PR — it produces the user-facing message and is covered by a test
§5.11 Aliased Array	Yes	Mapping builds new arrays; no in-place mutation of inputs
§5.12 Client-Only Guard	Yes	Host allowlist, game restriction, and format detection all enforced server-side

Testing

Network is dependency-injected; no test performs a live fetch.

• URL validation: rejects http://, non-allowlisted hosts, archidekt.com.evil.com, https://archidekt.com@evil.com/, non-numeric ids, malformed URLs; accepts archidekt.com and www.archidekt.com
• Mapping: against a trimmed fixture of the real captured response — asserts the two trap fields (card.collectorNumber path, uppercased editioncode)
• Maybeboard: a card in an includedInDeck: false category is excluded; one in Sideboard (true) is kept
• Transport: 404, 429, timeout, and refused-redirect each map to their own message; oversized response rejected
• Moxfield: a Moxfield URL returns the specific disabled message
• Detection: ManaBox and TCGplayer headers detect correctly; an unknown header errors
• Schema: exactly-one-of rawText | lines | sourceUrl; game required alongside sourceUrl; oversized/invalid URL rejected
• Client: four mode buttons render; upload sends format: 'auto'; URL mode sends sourceUrl

Bars: npm test, npm run test:client, npm run lint, npm run build all clean; ≥70% coverage on every touched file.

Follow-ups (not in this work)

• Brent to email Moxfield support requesting an allowlisted user-agent; enabling the connector is then a flag flip plus a mapper
• If Archidekt's rate limiter proves tight in practice, consider a short-TTL cache keyed on deck id