Notes

A collections of notes on papers, tech talks and articles that I’ve found interesting.

My goal is to extract the ideas I found particularly interesting so I can better internalize and remember them.

consensus

Raft: In Search of an Understandable Consensus Algorithm [Stanford, 2014]

database

Bigtable: A Distributed Storage System for Structured Data [Google, 2006]

Cassandra: A Decentralized Structured Storage System [Facebook, 2009]

Dynamo: Amazon's Highly Available Key-value Store [Amazon, 2007]

F1: A Distributed SQL Database That Scales [Google, 2013]

Mesa: Geo-Replicated, Near Real-Time, Scalable Data Warehousing [Google, 2014]

PNUTS: Yahoo!'s Hosted Data Serving Platform [Yahoo!, 2008]

Spanner: Google's Globally-Distributed Database [Google, 2012]

TAO: Facebook’s Distributed Data Store for the Social Graph [Facebook, 2013]

distributed computation

Dremel: Interactive Analysis of Web-Scale Datasets [Google, 2010]

FlumeJava: Easy, Efficient Data-Parallel Pipelines [Google, 2010]

Hive: A Warehousing Solution Over a Map-Reduce Framework [Facebook, 2009]

MapReduce: Simplified Data Processing on Large Clusters [Google, 2004]

Percolator: Large-scale Incremental Processing Using Distributed Transactions and Notifications [Google, 2010]

Tenzing: A SQL Implementation On The MapReduce Framework [Google, 2011]

experimentation

Trustworthy Online Controlled Experiments: Five Puzzling Outcomes Explained [Microsoft, 2012]

graph

GraphChi: Large-Scale Graph Computation on Just a PC [CMU, 2012]

PowerGraph: Distributed Graph-Parallel Computation on Natural Graphs [CMU, 2012]

Pregel: A System for Large-Scale Graph Processing [Google, 2010]

peer-to-peer

Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications [Stanford, 2001]

search

Approximating the Top-m Passages in a Parallel Question Answering System [UWaterloo, 2004]

Earlybird: Real-Time Search at Twitter [Twitter, 2012]

Unicorn: A System for Searching the Social Graph [Facebook, 2013]

Web Search for a Planet: The Google Cluster Architecture [Google, 2013]

storage

Erasure Coding in Windows Azure Storage [Microsoft, 2012]

Finding a needle in Haystack: Facebook’s photo storage [Facebook, 2010]

GFS: Evolution on Fast-forward [Google, 2009]

RCFile: A Fast and Space-efficient Data Placement Structure in MapReduce-based Warehouse Systems [Facebook, 2011]

The Google File System [Google, 2003]

XORing Elephants: Novel Erasure Codes for Big Data [USC & Facebook, 2013]

streaming

Kafka: a Distributed Messaging System for Log Processing [LinkedIn, 2011]

MillWheel: Fault-Tolerant Stream Processing at Internet Scale [Google, 2013]

Photon: Fault-tolerant and Scalable Joining of Continuous Data Streams [Google, 2013]

systems

The Case for Energy-Proportional Computing [Google, 2007]

The Five Minute Rule for Trading Memory for Disc Accesses and the 5 Byte Rule for Trading Memory for CPU Time [Tandem Computers, 1987]

The Five-Minute Rule 10 Years Later, and Other Computer Storage Rules of Thumb [Microsoft, 1997]

The Five-Minute Rule 20 Years Later (and How Flash Memory Changes the Rules) [HP Labs, 2009]

virtualization

Beyond Server Consolidation [Amazon, 2009]