Why Dropbox decided to drop AWS and build its own infrastructure and network
There is always a tension inside companies about whether to build or to buy, whatever the need. A few years ago Dropbox decided it was going to move the majority of its infrastructure requirements from AWS into its own data centers. As you can imagine, it took a monumental effort, but the company believed that the advantages of controlling its own destiny would be worth all of the challenges they faced to get there.
For starters, a company like Dropbox is dealing with a huge number of customers storing an enormous amount of data. The latest numbers are 500 million users and 200,000 business customers. When they made the transition, they had to move an epic 500 petabytes — that’s five followed by 17 zeros — that had been sitting on AWS servers. (They still use AWS for some workloads.)
The first step was building the infrastructure to replace it. We’re talking about a company that had 1500 employees, with just around a dozen on the infrastructure team. This was not a huge operation, yet what they were trying to do was build something themselves at web scale that only a small number of companies with much larger teams had tried to this point.
That included building and equipping three US data centers. It also meant building the network backbone, the infrastructure that facilitated the connections between the US data centers and other facilities they had located throughout the world. When you open Dropbox and request a file, you want your file to download pretty much instantaneously without latency, and it was up to the team to ensure that happened, while trying to navigate between the old system and the new one.
“What’s neat about the backbone is that it’s similar to something you might find at Google or Facebook, but we built this with a relatively small team,” Dan Williams, head of production engineering at Dropbox told TechCrunch. And that small team had to build the backbone and move all of that the content, all while keeping the service up and running.
Ultimately though the company was willing to make this massive move because it craved control over its infrastructure. Williams admitted that when it comes to making a decision like this, there are always concessions, but in the end the trade-offs were worth it to them. “For us, it was about quality and control and management. We know there are solid third parties out there with [high] quality and performance, but we felt ours could be equal or even better because we know the system so well.”
Williams says for Dropbox, building the network was a business decision and it has had a positive impact on the business overall. “I think it could be argued in fact that anyone who has built a decent-sized network like this has had some effect on the business in a positive way that is actually building trust for the user and getting more users to adopt the product or service based on the quality of the service” Williams explained.
The new system has certainly had a positive impact on Dropbox’s reputation with enterprise IT too. Back in the day, Dropbox often had a bad rep with IT because of unauthorized usage inside large organizations. Today, the Dropbox Business line of products combined with this in-house infrastructure and network has created a level of trust they didn’t have before. Williams points out that they have such detailed insight into the networking operation, they can use that data as a sales driver, even though they didn’t end up charging for this change.
“One of our core beliefs is providing a high-performance, low-expense product.” He believes that’s why they have been able to retain old customers through this transition, while also growing the user base over time.
Whether the cost of ownership went down in real terms, the company believes it achieved its objectives by building it themselves. “While cost is always something that we consider, our goal with our network expansion was to improve performance, reliability, flexibility and control for our users — which we have succeeded in doing,” a company spokesperson told TechCrunch.
That build versus buy decision is never an easy one, especially for a company the size of Dropbox, which had to walk the line between the two systems while it made the transition, but today it appears to have paid off in a big way for them.
Featured Image: kynny/Getty Images