Smooth like btrfs: Inside Facebook's Linux-powered infrastructure

Smooth like btrfs: Inside Facebook's Linux-powered infrastructure

Btrfs creator showcases Facebook's open-source storage.

Facebook engineer Chris Mason is unequivocal about the primacy of Linux in Facebook's storage infrastructure.

"If it runs on a computer, and it's storing important data," he said, "it's running Linux."

Mason, speaking at the Linux Enterprise End-User Summit on Monday in New York, joined Facebook just six months ago in order to spearhead the social network's move to btrfs (usually pronounced "butter eff ess."), the Linux-based file system that he created in 2008 while working at Oracle.

It's a massive undertaking, to say the least. Mason cited some statistics from Facebook's Lookback video project that help underline the scale of many Facebook plans -- 11 petabytes of storage used, a peak outgoing bandwidth rate of 450Gbps.

What's more, the pace of work at the company is often breakneck -- the code behind the main website itself is updated at least twice a day, according to Mason. "Projects there move very fast," he said. "Very soon after I joined, they had a selection of machines carved out to run btrfs."

Accordingly, the ability to automate much of the management work involved in the storage infrastructure is crucial, Mason said.

"If it can't be automated, we're not interested," he said. "We have so many machines and so much infrastructure that we have to be able to sit at a desk and hit a button and have it go."

Another reason Mason and his team have been able to keep Facebook's massive storage infrastructure from sputtering during the transition is the use of a stable Linux kernel, without too many in-house patches applied to the official stable branch.

"We've never had real problems caused by stable release," he said. "Being so close to the mainline kernel allows us to add new features." Most Linux machines used at Facebook run on version 3.1 of the Kernel or above, and the internal distribution is based on CentOS.

That's not to say that the project has been hiccup-free, of course.

"It turns out that there are little things that happen when you move from one kernel to another," Mason said.

Btrfs is what's known as a copy-on-write file system, which means that it operates in a substantially different way than traditional filesystems. Mason told in 2008 that the idea is to let Linux storage scale more efficiently across larger volumes. For many, the Facebook adoption will look like vindication.

Facebook has a history of headline-grabbing moves into open-source technology -- the company's Open Compute Project, created to help spur open-source data center hardware design, launched in 2011, and Facebook contributes code to a host of open-source software projects.

Follow Us

Join the newsletter!


Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

Tags open sourceOracleFacebookLinuxoperating systemssoftwareOpen Source Subnet


ARN Innovation Awards 2022

Innovation Awards is the market-leading awards program for celebrating ecosystem innovation and excellence across the technology sector in Australia.

EDGE 2022

EDGE is the leading technology conference for business leaders in Australia and New Zealand, built on the foundations of collaboration, education and advancement.

Brand Post

Channel Roadmap

The Channel Roadmap is a bespoke content hub housing strategic priorities from technology vendors for 2022 and beyond, partners can find the guidance on the key technologies and markets to pursue, to help build a blueprint for future success.

Show Comments