About an hour before kickoff, Yahoo fantasy sports lost all functionality. No making roster changes, no adding free agents, no scoring, no nothing. That equaled a lot of pissed off users. By Monday morning, things were largely fixed?small consolation to anyone who got shafted the day before because they couldn't set their teams. Yahoo sent out a letter of apology that didn't actually shed much more light on what went wrong.
A second letter was emailed to Yahoo users this afternoon, and provides some more detail:
Dear Yahoo! Fantasy Users,
As I mentioned in my earlier email, Yahoo! works hard to provide great experiences for our users every day. For you, we understand that Fantasy is a core part of your daily routine during this time of year. On Sunday we experienced technical issues which caused the Fantasy experience to be inaccessible. We're sorry. Our entire team worked tirelessly to re-establish the service - however, we had a significant outage. We know this created inconvenience and frustration for millions of our fans, and we feel we owe you an explanation.
So what happened? At Yahoo!, we have giant machines called "filers" that process a lot of the real-time data and stats for us and for you. We do millions of calculations every hour for our games, and normally our machines can handle this with no problem. Recently, we discovered a hardware issue in one of the filers that caused the other one to overload. We replaced some hardware, re-configured the setup, and did some testing. However this Sunday - at approximately 12:15 p.m. Eastern - the new configuration failed. This created an overload on storage capacity and took the Fantasy part of our site down.
We had dozens of engineers from various teams working together to try to determine the cause and fix it. One option was to fall back on another data center, but that would have meant shutting the game down and losing scoring data. We wanted to avoid that at all costs. Ultimately, we were able to move our mobile apps to a back-up data center, free up storage to get the PC version of the game working, and get the mobile apps up in a "read-only" state - meaning you could see scores and data, but you still couldn't set lineups and interact.
We spent Sunday night and most of Monday looking at dozens of potential causes. Monday afternoon, we stress-tested our system. Everything seemed to be in working order, so we turned on all our mobile app functions in time for Monday Night Football. Everything performed as expected and continues to do so. We'll have all hands on deck this coming Sunday to closely monitor performance and ensure we can respond quickly in case of any abnormal activity.
We know many of you have questions about your leagues and games, so we've also set up a FAQ here that will explain how we are going to handle various scenarios.
Finally, we know we failed you this weekend, and there's no way to fix that. As a token of our appreciation for your patience and continued support, we wanted to give something back. First, we're offering a complimentary football weekly Scouting Report for the remainder of the year. Hopefully this will help you down the stretch. The report includes weekly expert analysis, rankings and news to help you make smart decisions. Second, we are offering all our users a 20% discount for the rest the year at the Yahoo! Sports Store.
For any updates, we will keep you informed on @YahooSports and facebook.com/yahoosports.
We won't stop working to provide you with new and improved features that help you enjoy your Sundays.
Thanks again for playing and for your patience.
Best,
Ken Fuchs
Head of Yahoo! Sports
In your individual leagues, the commissioner has the power to let the Week 10 results stand, or vacate them entirely. Either option is sure to piss someone off.
world war z Johnny Manziel the voice When Is Veterans Day 2012 brooke burke jennifer lawrence Alexa Vega
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.