Eventual State Consistency vs. Eventual Path Consistency

Overview

Brewer’s CAP Theorem says that a distributed application has to relax at least one of three properties - Consistency, Availability and Partition Tolerance.

It is often Consistency that is relaxed in favor of Availability and Partitioning.  For example, several popular database engines provide for geographically distributed instances, each of which commits its transactions locally, and then replicates to one or more remote instances.  In the event these updates collide with (or conflict with) updates made locally, corrective action can be initiated, either with a blanket rule such as “local always wins”, “remote always wins” or “ignore”; or with something more sophisticated, such as invoking application logic to take corrective action.

However, there are applications where detection of colliding updates does not suffice to ensure Consistency (where Consistency is defined as the database always being within the design parameters of the application logic – for example, the “in balance” guarantee that the sums of the assets and liabilities columns of a balance sheet are always equal).  There are applications where Consistency may require that the data read by application logic, not just the updates. must also be validated when checking for collisions.

I refer to collision detection by validating updates alone as Eventual State Consistency, and collision detection by also validating data read as Eventual Path Consistency.

From 44 seconds to 27 seconds - Simple Tuning of the 3n+1 Benchmark on GT.M

The bottom line first: with a little simple tuning, I was able to improve the performance on my laptop of my benchmark of the 1 to 1,000,000 3n+1 problem.

• For four worker processes:
• from 44 to 27 seconds
• from to 72,026 to 109,311 reads/second
• from to 49,298 to 74,828 updates/second
• For eight worker processes:
• from 248 to 30 seconds
• from 12,782 to 109,285 reads/second
• from 8,750 to 74,802 updates/second

Responses to Dan Weinreb's comments of February 1, 2011

I am responding to Dan Weinreb's comments to my post Comments in response to Don Anderson's 3n+1 benchmark with another blog entry because this was too long to post as a reply.

Comments in response to Don Anderson's 3n+1 benchmark

I am thrilled that Don Anderson has responded to a benchmarking problem I proposed aimed at apples to apples benchmarks of databases and platforms.

Results for 3n+1 Benchmark version 2 for GT.M

3n+1 NoSQL/Key-value/Schema-free/Sche... - version 2

Problem statement of the second version of the benchmark.  An earlier version of this benchmark problem statement can be found at
http://ksbhaskar.blogspot.com/2010/02/3n1-nosqlkey-valueschema-freesche.html

Motivation

This is an attempt to create a benchmark for apples to apples comparisons of databases that are generally described as NoSQL, key-value, schema-free or schema-less databases.  The goals are to create a benchmark that:

• is simple to understand and code in most programming languages;
• performs a non-trivial amount of representative activity - reads and updates, access contention, recovery, etc.; and
• generates results that can be easily replicated and verified by others.

Encrypting Partitions on GNU/Linux

Motivation

Encrypting partitions helps protect against unauthorized access to data in the event the computer hardware is physically lost or stolen. The technique described here relies on the ability of the Linux kernel to:

1. Create an a map of a real device or a logical volume.
2. Create an encrypted block device
   

Enhanced 3n+1 Benchmark for GT.M - 2010-02-28

3n+1 NoSQL/Key-Value/Schema-Free/Schema-Less Database Benchmark

Motivation

This is an attempt to create a benchmark for apples to apples comparisons of databases that are generally described as NoSQL, key-value, schema-free or schema-less databases.  The goals are to create a benchmark that:

• is simple to understand and code - ideally, a page of code or less in most programming languages;
• performs a non-trivial amount of representative activity - reads and updates, access contention, recovery, etc.; and
• generates results that can be easily replicated and verified by others.