Now taking bets on which IT vendor will push database technology beyond the bleeding edge. Yesterday, 10.5 billion shares traded hands between 9:30 AM and 4:00 PM. Double that to cover both sides of the trade and divide by 100 to get round lots: 200 million trade-sides in round numbers. That’s about 8,900 transactions per second — average. Peak could be 10x: 90,000 transactions per second. (In 1990, a busy bank ATM system was 10 transactions a second).
Put all of the following into a relational database, and allow simultaneous complex queries by six federal agencies (multiples users per agency):
institution, account, trade type, security CUSIP number, shares, price, date-time (UTC, to the microsecond), funds type (margin bears more scrutiny than cash), execution exchange, routed exchange … and more. Keep running totals and details in real-time of 10,000 securities, 5,000 institutions, and selected accounts. Capture all this data at the exchanges without slowing down exchange traffic, and ship it to a very secure location near Wall Street (e.g., Hoboken, NJ) for loading into the database. Figure 100 gigabytes a day. Since this will be the government’s database of record, the system has to be fault tolerant, secure, and auditable.
You have $4 billion budget and 3 years to get it up and running. Be prepared to testify frequently to Congress. Gentlemen and Ladies, start writing your proposals.
The SEC comment period is open for sixty days. I’m fascinated by how this system will be architected, and not convinced it can be successfully built.