Cosmic Conservative

Rational Conservatism from the heart of America

Denial of tickets attack

By CosmicConservative
Rockies ticket sales were down due to an external "malicious attack."
FOXNews.com - World Series Ticket Sales Suspended After 'Malicious Attack' Crashes Computers - Local News | News Articles | National News | US News DENVER — Colorado Rockies officials said Monday their computer system for online-only World Series ticket sales was the target of an "external malicious attack," but online ticket sales to resume at noon Tuesday.
The story sort of implies a "denial of service" attack. For a company that provides online commerce for selling sporting event tickets, I would expect them to have firewalls that shut down traffic on any ports not needed for their legitimate traffic, load balancers and scalable web server infrastructure, with additional load balancers at the app server level and, if they use them, even at the database server. Such an infrastructure should be able to handle most amateur denial of service attacks simply by restricting traffic to the narrow band that is legitimate, and then essentially out-muscling the requests. To defeat a truly enterprise level internet architecture requires some serious muscle. Usually that muscle is assembled by the denial of service attacker recruiting hundreds or even thousands of external PCs to flood the site with page requests. This is usually done by utilizing home PCs infected by worms which listen for a broadcast command and then send page requests to the target in the background without the user even knowing it is happening. Some worms have infected tens of thousands of computers and the IP addresses of those infected computers are made available to "hackers" for this purpose, sometimes at a price. So if that was what happened, it should be easy to show that such a technique was used. That technique is virtually impossible to attempt without using computers across many states (and countries). So since this is an e-commerce site, and the attack would have crossed state boundaries, if this is true, then such a denial of service attack is a federal crime. And such attacks frequently leave traces. I'd love to know more about this.... Just call it professional curiosity....

Comments (7)

CosmicConservative
Oh, one of the most common public "explanations" for server crashes of this sort is that it was a "malicious attack" but such massive coordinated attacks are not that common. And when they happen, they generally target much more well-known targets, like the US Government, Microsoft, Google, Amazon.com, etc. My guess is that the servers got overloaded, this caused response times to slow down, and users got irritated and started pounding their mice buttons or refresh buttons, and that sent dozens of requests per user and the ticket websites were overwhelmed by the resulting flood of traffic. This is actually pretty common and does sort of look like a DOS to a casual observer. In my previous job we had several outages over the years that made us suspicious of being attacked, but only one that I know of was ever more or less proven to be malicious. All of the rest ended up being the result of misconfiguring our load balancers, releasing code with a bug, or (most common) database problems that created a bottleneck between the app servers and the database servers. I really think that is likely the case here too. If so they can "fix" it by optimizing the offending queries, reconfiguring their infrastructure, or simply by adding servers to their web and/or app clusters. This is pretty simple, if your architecture is scalable. My suspicion is that is what they did, and it usually takes a couple of hours to install and configure additional servers, assuming your network infrastructure can handle it (which it usually can). In one of our major releases we had a similar problem and we grabbed a bunch of development and test servers that we had ordered for a different project and threw them into the data center, configured them and had them up and running overnight. As I recall we threw something like 40 servers at the problem and that fixed it. Of course then we had to go purchase replacements for the "borrowed" servers... Since it took them a full day to respond and fix the problem, I believe that they had to go buy some servers and install them, and the longest delay was probably putting the servers in a truck and driving them to the data center.
CosmicConservative
One more comment. A caller into KOA said that they were one of the first people to try to buy tickets, ran into a problem and called the company to report the problem. They were told that the site does not work with FireFox, that they had to use Internet Explorer. If that is true then that would explain why they had a flood of hits since all the Firefox users (and that's about 35% of the user base the last time I checked) were probably rapid-fire punching their "refresh" buttons.
CosmicConservative
So, a separate issue from this, but a troubling one... On the night before Game 7 of the ALCS, a major west coast newspaper broke the story of the Cleveland Indian's game 7 pitcher having taken human growth hormone, which is now a banned substance for MLB. The source of the story is supposedly the investigation into steriod use that ex-Senator George Mitchell is leading up. George Mitchell is a high-up official in the Boston Red Sox organization. This is very troubling to me. It stinks of using information gained in a supposedly confidential federal investigation to influence the outcome of the championship game leading up to the world series....
Andy
George Mitchell? Come on...this guy is a pussycat...how could he ever be involved with a leak to the SF Chronicle (gagging right now)? This guy is the consummate phony, and has always been. He denied it (big surprise): http://sportsillustrated.cnn.com/2007/baseball/mlb/wires/10/22/2010.ap.bba.indians.byrd.hgh.0667/
CosmicConservative » It actually IS a federal crime
[...] Remember when I said that if the Rockies World Series website had actually been the target of an attack that it would be a federal crime? Apparently the FBI agrees with me. DENVER (AP) - The FBI opened an investigation after the Colorado Rockies claimed an “external, malicious attack” on computers halted the team’s first attempt to sell World Series tickets. [...]