Machine Learning & LLM

This is a long-form judgment piece in the "Machine Learning & LLM" series: Can LLMs really do trading? We skip the concepts and focus only on three experiments we conducted, the pitfalls we encountered, and an honest conclusion.

One-Sentence Conclusion

Question	Our Answer	Site Entry
Can LLMs directly place orders to make money?	No (not yet, at least)	Review of Three Experiments
What can LLMs do?	Research report analysis, code generation, factor ideation	RD-Agent, Hermes Live Test
Is traditional ML still worth learning?	Yes, XGBoost remains the king of tabular competitions	PCA + Wavelet + XGBoost Paper Reading

I. Three Experiments (We Did Them Ourselves)

Can LLMs Do Trading? Three Experiments Explain the Principles is one of the most-read experimental articles on this site. Here are the conclusions upfront:

• Experiment 1: Let LLM directly predict price movements — On the CSI 1000 index, the results were roughly equivalent to random guessing. LLMs understand numbers at the "token level," not the "numerical level" (BPE tokenizes 123.45 into pieces, so it doesn't see continuity).
• Experiment 2: Let LLM write strategy code — This actually works. If you describe the requirements clearly, the backtesting framework it generates is usable. It saves time on "writing boilerplate code."
• Experiment 3: Let LLM read research reports to extract factor ideas — Partially works. It can extract "testable hypotheses" from reports, but you must verify each hypothesis yourself.

Taken together, the three experiments boil down to one sentence: LLMs are excellent research assistants, not traders.

II. The Status of Traditional ML: XGBoost Is Still Competitive

LLMs are all the rage, but for tabular data, the GBDT family remains the king of cost-performance:

• Why XGBoost Is Always Competitive — Second-order Taylor expansion + regularization. This mathematics is hard to beat on small-to-medium-sized tabular datasets.
• PCA + Wavelet + XGBoost Paper Reading — Feature engineering + GBDT remains the mainstream recipe in papers.
• XGBoost Custom Objective Functions — Understanding the requirement for "second-order differentiability" is necessary to write custom objectives.

Advice for beginners: Master XGBoost first, then talk about LLMs. LLM strategies that can't even beat tabular baselines are not worth deploying in live trading.

III. Real Progress in the Agent Direction

Agents (multi-agent collaboration) are the hottest direction for 2025–2026. We have followed three tracks:

• Microsoft RD-Agent — This breaks "factor mining" into a four-agent "hypothesis-verification" loop (R&D Loop). It is currently the Agent framework that most resembles a "quantitative researcher." It is worth running its Factor Task.
• Hermes Live Test — We installed it ourselves and encountered pitfalls. The conclusion is: "Good ideas, but engineering is still early." It is suitable for experimentation, not production.
• Vibe Coding for Large Projects — Testing the limits of Agent code writing: The conclusion is that small, iterative steps work, but one-time large requirements do not.

Taken together, the three tracks suggest: Today, the most suitable role for Agents is that of an "tireless junior researcher." Give it clear task definitions and verification standards, and it can run hypotheses 24/7; but directional judgment and final decisions still rest with humans.

IV. Advice for Different People

• Beginners: Don't chase LLMs. First, learn Numpy/Pandas and XGBoost. If the foundation isn't solid, you won't be able to verify the outputs from Agents.
• Those with a foundation: Run the Factor Task in RD-Agent to understand how the "hypothesis-verification" loop can be automated.
• Researchers: Use LLMs as "research report readers + code assistants," and retain manual verification steps. Focus on "how to verify" rather than "how to generate."

V. Further Reading

• Can LLMs Do Trading? Three Experiments — Required reading; full text of the three experiments.
• Microsoft RD-Agent — Agents doing factor mining.
• Hermes Live Test — Current state of Agent engineering.
• XGBoost Second-Order Information and Regularization — The mathematics behind the king of tabular data.
• PCA + Wavelet + XGBoost — The recipe for papers.
• Option Pricing Without Writing Code — The other side of AI-generated strategies.

One sentence: LLMs have changed "how research is done," but they have not changed "what good research is." The verification standards remain the same; only execution has become faster.