QCon 2017: The Power of the Log

VIDEO HERE

This talk is about the beauty of sequential access and append only data structures. We’ll do this in the context of a little known paper entitled “Log Structured Merge Trees”. LSM describes a surprisingly counterintuitive approach to storing and accessing data in a sequential fashion. It came to prominence in Google’s Big Table paper and today, the use of Logs, LSM and append only data structures drive many of the world’s most influential storage systems: Cassandra, HBase, RocksDB, Kafka and more. Finally we’ll look at how the beauty of sequential access goes beyond database internals, right through to how applications communicate, share data and scale.

The Power of the Log from Ben Stopford

Posted on March 8th, 2017 in Uncategorized

No comments

Jump to comment form | comments rss

Have your say

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

Talks (View on YouTube)

Essays (all)

The Data Dichotomy (2016)
The Benefits of “In-Memory” Data are Often Overstated (2016)
Elements of Scale: Composing and Scaling Data Platforms (2015)
Upside Down Databases: Bridging the Operational and Analytic Worlds with Streams (2015)
Log Structured Merge Trees (2015)
Building a Career in Technology (2015)
A World of Chinese Whispers (2014)
Database Y (2013)
The Big Data Conundrum (2012)
Where does Big Data meet Big Database? (2012)
A Story about George (2012)
The Rebirth of the In-Memory Database (2011)
Is the Traditional Database a Thing of the Past? (2009)
Shared Nothing v.s. Shared Disk Architectures: An Independent View (2009)
Component Software. Where is it going? (2005)
Do Metrics Have a Place in Software Engineering Today? (2004)

Test Driven Development (all)

Test Oriented Languages: Is it Time for a New Era? (2011)
Beyond Stubs: Why We Need Interaction Testing (2010)
Isolating Functional Units: Why We Need Stubs (2010)
Are Mocks All They Are Cracked Up To Be? (2010)

Coherence (all)

About

Twitter, RSS, Github, Photography, Full Bio.

Data Tech (all)

Best of VLDB 2014 (2015)
A Guide to building a Central, Consolidated Data Store for a Company (2014)
An initial look at Actian’s ‘SQL in Hadoop’ (2014)
The Best of VLDB 2012 (2012)
Thinking in Graphs: Neo4J (2012)
A Brief Summary of the NoSQL World (2012)
ODC – A Distributed Datastore built at RBS (2012)
Looking at Intel Xeon Phi (Kinghts Corner) (2012)

Team / Process / Interviewing (all)

The Iffy Tractor (Can they code OO?) (2011)
The Business Analyst Test (2011)
Distributing Skills Across a Continental Divide (2011)
Learning Practices for Distributed Teams (ICST) (2011)
Interviewing: The Importance of Examining Applied Knowledge (2010)
Mapping Personal Practices (2010)
Four HPC Architecture Questions – With Answers (2009)