Trading Agent Data · Research
Give an AI model the filing date, or it will assume the figure was already known
Asked what a company had filed by a given day, 4 of Anthropic's Claude models were right when each figure carried its filing date. Given the same figures without it, they treated the end of the period as the day the figure became known, and answered with numbers that had not been filed yet.
866 questions, 10 markets, run on by WU Capital Limited, which publishes Trading Agent Data. The scores are the models' answers to our questions. Read the limits before quoting them.
What we asked
Figure 1. What the SEC had on file for Apple, by day (SEC EDGAR, read from our API on 19 November 2026). Asked about 2 November, a model given the filing dates answers with the June quarter, US$109.4bn, which is right: the full-year figure was filed a day later. A model given the same rows without filing dates sees a year that ended on 26 September and answers US$478.8bn. Every question in the test has this shape. We ran each one twice: with a filing date on every row, and with that column and the two instructions that mention it taken out.
What the models did
Show the numbers as a table
| Model | Correct, with | Correct, without | Not-yet-filed figure, with | Not-yet-filed figure, without |
|---|---|---|---|---|
| Claude Fable 5.2claude-fable-5-2 | 100.0% | 72.5% | 0.0% | 62.6% |
| Claude Opus 5.5claude-opus-5-5 | 100.0% | 73.9% | 0.0% | 53.5% |
| Claude Sonnet 5.5claude-sonnet-5-5 | 99.7% | 68.5% | 0.0% | 76.5% |
| Claude Haiku 5.5claude-haiku-5-5 | 97.5% | 60.5% | 1.2% | 97.9% |
Figure 2. 866 questions per model and version, one run each, 19 November 2026, covering 101 companies and charities. "Correct" is the share of all questions answered with the figure the answer key holds: 97.5% to 100.0% with filing dates. The last column is the share of the 340 trap questions (a period had ended, its figure was filed later) answered with that later-filed figure: 53.5% to 97.9%. Without filing dates no model declined to answer more than 7 times. A third version, with the filing-date column but without the instructions on how to use it, scored the same as the first to within a few questions.
What to do about it
- Check whether your data has a filing date on every figure. A period end is not one. If the date is missing, anything that asks about a past day is exposed: a backtest, a review of a past credit decision, an agent asked what was known when a contract was signed.
- Give the model the date and the day you are asking about. You do not need to teach it the rule. With the column present, the models used it correctly without being told how.
- Do not count on the model to warn you. When the date was missing, the models answered anyway. Build the check into the data, not into the prompt.
- Ask your data supplier what its date means. Registers record the day of submission, the day of publication, or neither. Ours are set out market by market in the methodology.
- See it on one company. The guide to point-in-time data has a query that runs on the public demo key and returns Apple's revenue as it stood on 2 November 2026. To see which companies on your own list carry filing dates with us, use the coverage check.
Limits
- Only Anthropic's Claude models were tested, with one run per model and version. Other providers' models may behave differently, and scores can move by a few questions between runs.
- The version without filing dates measures an assumption, not ability: with no date in the table there is no way to be sure of the answer. It shows what a model does with such data, not that the model is weak.
- The questions are built from our own data, and the first version is easy for current models by design: it states the rule and gives the date.
- Some replies in the version without filing dates did not follow the answer format and were scored as wrong (at most 19 of 866 for one model).
- This is a test of how models read dated records. It says nothing about any company and is not investment advice.
How it was scored
A program compares each reply with the answer key. A reply is correct if it gives the figure the key holds for that question. It counts as using a not-yet-filed figure if it gives a figure from the same table whose filing date is after the day asked about. Declining to answer is counted separately. The models were called through their provider's API with its adaptive reasoning setting and no system prompt, and saw only the question and the table.
Questions about this test: contact@tradingagentapp.com.
Related
- What is point-in-time fundamentals data?
- Filing Observatory: how long after the year end companies file.
- AI agents and platforms: using the data from an agent.
- API reference.