Craig Pirrong, the Streetwise Professor, recently wrote about his skeptical take on the CFTC’s desire to examine the source code of trading algorithms. The proposed Regulation AT (Automated Trading) has many issues, and Pirrong calls out two of them:
I seriously doubt that the CFTC can attract people with the coding skill necessary to track down errors in trading algorithms, or can devote the time necessary… for a truly effective review.
This is a great point. If the CFTC is so burdened with what’s on their regulatory plate already, how can they possibly add this? And how can the CFTC hope to compete with trading firms for the technical talent required to effectively review such code?
Second, and more substantively, reviewing individual trading algorithms in isolation is of limited value in determining their potentially disruptive effects…
This is because in complex systems, attempts to improve the safety of individual components of the system can actually increase the probability of system failure.
Pirrong is a scholar after our own hearts, and he hits on so many important points here. The theory of complex systems tells us that non-holistic safety mechanisms often make things worse.
For example, after the 2010 Flash Crash, the SEC implemented single-stock circuit breakers. Such measures seem like a good idea, and the circuit breakers often help minimize disruptions. But on August 24, 2015, these single-stock circuit breakers halted trading in 471 different ETFs and stocks. This in turn lead to further dislocation as many key ETF liquidity providers simply stopped trading because they could no longer model the baskets of securities that underlie many ETFs.
Worse, if the intent is to prevent Knight-like fiascos, the CFTC should look elsewhere. Knight’s problem wasn’t even a coding error. Knight’s code worked—it was just deployed incorrectly. If that sounds like splitting hairs, that’s precisely the point. These systems are so complicated that code divorced from configuration files and deployment procedures is essentially meaningless.
I understand where the desire for a Reg AT-type solution comes from. The complexity of the financial markets is increasing, and we’ve seen over and over that regulators are struggling to get a handle on things. But if the CFTC really wants a window into the risk of automated trading, they should take a page from the Federal Aviation Administration’s playbook (as we’ve argued before). The FAA supports the airline industry’s quest for safety by cooperatively interfacing with airline-run Safety Management Systems. These systems specify a structure for reporting, discussing, correcting errors, and for auditing those corrections—largely without the fear of regulatory reprisal.
The CFTC should drop the costly, draconian, ultimately counterproductive Reg AT proposal. Instead, they should consider “Reg SMS,” in which they work with the industry to set up standard error capture, discussion, and QA processes—modeled after the airlines’ Safety Management Systems—so we can all get a handle on this complexity together.
Just as there are best practices for coding, there are best practices for managing complexity. The CFTC needs to look for them.