Systems Lab

Research and corrections policy

Cheetah Systems Lab is maintained by the Cheetah research team. The team is accountable for the selection rules, source labels, research dates, corrections, and publication gates used in the directory.

How the research works

  1. Resolve the product identity and retain the official website before evaluating product claims.
  2. Prefer official documentation, API references, pricing pages, changelogs, status pages, and vendor-owned repositories.
  3. Label vendor claims, documented facts, and editorial assessments separately.
  4. Record a checked date and state missing evidence as not retrieved, never as proof that a feature does not exist.
  5. Do not claim runtime testing unless the product was actually run. Directory listings are document-based; where a page carries a named operator account, that product was run in production and the claim is labelled as first-hand.
  6. Require every published figure to appear in the source quote cited as its evidence, and treat automated extraction as a draft rather than a finding.

Automated collection, and what we ignore

Pages are fetched by an automated collector. Everything it retrieves is treated as data to be quoted, never as instructions to be followed. This matters more than it used to: vendor sites in this corpus now ship text written for AI agents rather than for readers. Three examples encountered during the 2026-08-27 research run — a pricing page directing agents to fetch and follow separate credential and onboarding files, a documentation site opening every page with a retrieval instruction, and a marketing site carrying the same instruction inside a hidden element. None were acted on.

A related practice is worth naming because it shapes what AI assistants report: some vendors publish a machine-readable pricing page addressed to AI agents alongside the human one, and the two do not always agree. Where that happens, both are recorded and the disagreement is stated on the page rather than resolved silently.

Automated extraction of figures from a page is a language model reading a document, and it has returned confident prices that appear nowhere on the page it read. Extracted figures are therefore verified against the retained page text before publication, and a figure that cannot be traced to a quote does not ship.

Alternatives-page policy

Competitive sets are editorial classifications, not rankings. Each set exposes its membership test and exclusions. A page may publish only when it has a researched anchor, at least 300 words, and more than 40 percent corpus-unique six-token shingles after global navigation and footer text are excluded.

Official links on these pages are a page-level source set unless a claim names a more precise source. The Lab does not present page-level links as field-by-field verification. Pricing appears only when the retained record says official pricing was retrieved.

Editorial review and corrections

Programmatic pages pass automated route, metadata, schema, internal-link, source-count, punctuation, word-count, and uniqueness checks. Representative pages from every competitive set are then reviewed for clarity, fairness, evidence labels, and visible research limits before publication.

If a source changes or a statement is wrong, send the exact page URL and correction through theCheetah contact form. The research team checks the official source, updates the retained record and checked date when warranted, then reruns the same publication gates. Material corrections are not silently converted into new claims.

Ownership and team background are available on the About Cheetah page. Research questions can also start in the Systems Lab help guide.

What a skill does to your systems

Each published agent skill carries a reading of what it has the agent do: advice only, read outside data, write to your systems, or contact people, plus whether it sends messages, writes records or spends credits, and which of HubSpot, Salesforce, Clay, Apollo.io and Instantly it operates. These labels are read from source, not tested by running. Nothing was executed.

  1. The SKILL.md is fetched at the commit the catalogue pins. Where the skill hands work to other skills in the same library, those files are read with it.
  2. A classification model (TypeSafe Jev, a fixed question set) answers each question with a probability. Only answers past a fixed confidence threshold are kept; everything else is shown as Unreviewed.
  3. Code finds candidate lines in the source and the model picks the one that best evidences each kept label. A label without a cited line is not published. “Reads outside data” must cite a line that names the data source or call, never a heading. “Advice only” is the absence of those steps, so it cites nothing and says so.
  4. A second, independent reviewer checks every label that says a skill writes, sends or spends, and every tool it is said to operate, against that cited line. Rejected labels are shown as Unreviewed.
  5. Whether a key or connector is needed comes from pattern matching on the files, not from the model.

The line is quoted only when the skill’s licence permits redistribution; otherwise the page links the file and line at the pinned commit. When a skill’s file changes, its reading is withdrawn until it is read again. On two blind samples of 60 skills, the kept consequence labels matched an independent rater’s first choice 85–87% of the time; treat them as a reading, not a guarantee.