HBaseCon2017 Accordion: Apache HBase Beathes with In-Memory Compaction
Modern products powered by HBase exhibit ever-increasing expectations from its read and write performance. Ideally, HBase applications would like to enjoy the speed of in-memory databases without giving up on the reliable persistent storage guarantees. We present Accordion - a new algorithm in HBase 2.0 that takes a significant step towards this goal. Accordion introduces Compacting MemStore - a MemStore implementation that periodically eliminate redundancies and other overhead while the recently written data is still in RAM. This reduces the frequency of flushes to HDFS, which simultaneously improves performance and write amplification. We present two flavors of the algorithm, and demonstrate their performance gains on a variety of workloads. Edward Bortnikov of Yahoo! Research, Israel
Modern products powered by HBase exhibit ever-increasing expectations from its read and write performance. Ideally, HBase applications would like to enjoy the speed of in-memory databases without giving up on the reliable persistent storage guarantees. We present Accordion - a new algorithm in HBase 2.0 that takes a significant step towards this goal. Accordion introduces Compacting MemStore - a MemStore implementation that periodically eliminate redundancies and other overhead while the recently written data is still in RAM. This reduces the frequency of flushes to HDFS, which simultaneously improves performance and write amplification. We present two flavors of the algorithm, and demonstrate their performance gains on a variety of workloads. Edward Bortnikov of Yahoo! Research, Israel