Automated monitoring, automated remediation workflows (for example, traffic shifting), and automated deployment systems are critical to detecting and resolving the vast majority of issues at this scale. However, for many reasons we still need to be able to see what these services, workflows, and deployments are doing at any moment in time
This guide covers timeless ideas that are helpful to keep in mind while working with systems where performance matters. Many of these ideas are fairly “durable” and will apply regardless of what hardware, programming language, operating system, or decade you are working in
In this post we will set out to establish a WireGuard tunnel between dynamically addressed peers that are both sitting behind a NAT. One of the primary goals for achieving this is to stick with WireGuard in its purest form, the code that now ships with the Linux Kernel. We do not want to compromise it in any fashion to achieve our goals, although we could get very creative with its user space implementation
This guide describes how to tune your AMD64/x86_64 hardware and Linux system for running real-time or low latency workloads.
Large-scale decentralized systems of autonomous agents interacting via asynchronous communication often experience the following self-healing dilemma: Fault-detection inherits network uncertainties making a faulty process indistinguishable from a slow process. The implications can be dramatic: Self-healing mechanisms become biased and cost-ineffective. In particular, triggering an undesirable fault-correction results in new faults that could be prevented with fault-tolerance instead. Nevertheless, fault-tolerance alone without eventually correcting persistent faults makes systems underperforming as well. Measuring, understanding and resolving such self-healing dilemmas is a timely challenge and critical requirement given the rise of distributed ledgers, edge computing, the Internet of Things in several application domains of energy, transport and health. This paper introduces a novel and general-purpose modeling of fault scenarios. They can accurately measure and predict inconsistencie
Edge computing moves the computation closer to the data and the data closer to the user to overcome the high latency communication of cloud computing. Storage at the edge allows data access with high speeds that enable latency-sensitive applications in areas such as autonomous driving and smart grid. However, several distributed services are typically designed for the cloud and building an efficient edge-enabled storage system is challenging because of the distributed and heterogeneous nature of the edge and its limited resources. In this paper, we propose EdgeKV, a decentralized storage system designed for the network edge. EdgeKV offers fast and reliable storage, utilizing data replication with strong consistency guarantees. With a location-transparent and interface-based design, EdgeKV can scale with a heterogeneous system of edge nodes. We implement a prototype of the EdgeKV modules in Golang and evaluate it in both the edge and cloud settings on the Grid'5000 testbed. We utilize the Yahoo! Cloud Serving Benchmark (YCSB) to analyze the system's performance under realistic workloads. Our evaluation results show that EdgeKV outperforms the cloud storage setting with both local and global data access with an average write response time and throughput improvements of 26% and 19% respectively under the same settings. Our evaluations also show that EdgeKV can scale with the number of clients, without sacrificing performance. Finally, we discuss the energy efficiency improvement when utilizing edge resources with EdgeKV instead of a centralized cloud.
Traditional approaches to replication require client requests to be ordered before making them durable by copying them to replicas. As a result, clients must wait for two round-trip times (RTTs) before updates complete. In this paper, we show that this entanglement of ordering and durability is unnecessary for strong consistency. Consistent Unordered Replication Protocol (CURP) allows clients to replicate requests that have not yet been ordered, as long as they are commutative. This strategy allows most operations to complete in 1 RTT (the same as an unreplicated system). We implemented CURP in the Redis and RAMCloud storage systems. In RAMCloud, CURP improved write latency by ~2x (14us -> 7.1us) and write throughput by 4x. Compared to unreplicated RAMCloud, CURP's latency overhead for 3-way replication is just 1us (6.1us vs 7.1us). CURP transformed a non-durable Redis cache into a consistent and durable storage system with only a small performance overhead.
State machine replication protocols, like MultiPaxos and Raft, are at the heart of nearly every strongly consistent distributed database. To tolerate machine failures, these protocols must replace failed machines with live machines, a process known as reconfiguration. Reconfiguration has become increasingly important over time as the need for frequent reconfiguration has grown. Despite this, reconfiguration has largely been neglected in the literature. In this paper, we present Matchmaker Paxos and Matchmaker MultiPaxos, a reconfigurable consensus and state machine replication protocol respectively. Our protocols can perform a reconfiguration with little to no impact on the latency or throughput of command processing; they can perform a reconfiguration in one round trip (theoretically) and a few milliseconds (empirically); they provide a number of theoretical insights; and they present a framework that can be generalized to other replication protocols in a way that previous reconfiguration techniques can not. We provide proofs of correctness for the protocols and optimizations, and present empirical results from an open source implementation.
In this post we'll design a break glass procedure for reaching SSH hosts in an emergency, using security keys that you can store offline. This is just one approach, but you can adapt it to your circumstances. We will store an offline SSH Certificate Authority on a hardware security key, and have our hosts trust that CA.
This is a short guide describing the latency implications of the virtual memory abstraction. If you are building systems requiring low and predictable latency such as realtime audio processing, control and high frequency trading (HFT) / algorithmic trading systems this guide will be useful to you. It is written from the perspective Linux kernel running on AMD64 / x86-64 architecture, but the general concepts applies to most operating systems and CPU architectures.
This paper presents a system called NetKernel that decouples the network stack from the guest virtual machine and offers it as an independent module. NetKernel represents a new paradigm where network stack can be managed as part of the virtualized infrastructure. It provides important efficiency benefits: By gaining control and visibility of the network stack, operator can perform network management more directly and flexibly, such as multiplexing VMs running different applications to the same network stack module to save CPU. Users also benefit from the simplified stack deployment and better performance. For example mTCP can be deployed without API change to support nginx natively, and shared memory networking can be readily enabled to improve performance of colocated VMs. Testbed evaluation using 100G NICs shows that NetKernel preserves the performance and scalability of both kernel and userspace network stacks, and provides the same isolation as the current architecture
State machine replication (SMR) is a well-known approach to implementing fault-tolerant services, providing high availability and strong consistency. To boost the performance of SMR, some proposals execute independent commands concurrently, while dependent commands execute sequentially in the total delivery order. The most general approach to handling command dependencies resorts to a directed acyclic graph (DAG), where nodes represent commands and edges represent dependencies. In this paper we show that due to the command arrival and multithreaded execution rates of SMR, a highly concurrent implementation of a DAG is needed. We show that a typical coarse-grained DAG implementation, where the whole graph is a critical section, results in a bottleneck in the replica. We propose two improvements to the coarse-grained DAG approach: fine-grained algorithms, using lock-coupling, and lock-free algorithms. Our fine-grain algorithms lock individual vertices in the DAG. The lock-free algorithms use nonblocking synchronization, with atomic operations, and lazy synchronization to postpone physical removal of nodes. All algorithms were integrated in a parallel SMR prototype. Experimental evaluation revealed that the fine-grained algorithms are also subject to a bottleneck. The lock-free implementation, however, sports linear speedup with the number of working threads, in some cases scaling up to 64 threads.
Barrierd offers the same functionality as membarrier(2)'s regular (non-EXPEDITED) asymmetric barriers. However, by tracking interrupts instead of waiting for a full RCU grace period, the barrier conditions are satisfied more quickly (on the order of 0.1-4 ms on my machine, rather than 25-80 ms).
The membarrier system call is used for synchronizing access to data structures shared by multiple threads. Its main use case is implementing synchronization primitives that can be split into fast and slow paths, for example the read-copy-update (RCU) algorithm. As the name implies, it's used to implement memory barrier semantics without actually using memory barrier instructions.
Back in February 2020, Blelloch and Wei submitted this cool preprint: Concurrent Reference Counting and Resource Management in Wait-free Constant Time. Their work mostly caught my attention because they propose a wait-free implementation of hazard pointers for safe memory reclamation.1 Safe memory reclamation (PDF) is a key component in lock-free algorithms when garbage collection isn’t an option,2 and hazard pointers (PDF) let us bound the amount of resources stranded by delayed cleanups much more tightly than, e.g., epoch reclamation (PDF). However the usual implementation has a loop in its read barriers (in the garbage collection sense), which can be annoying for code generation and bad for worst-case time bounds. Blelloch and Wei’s wait-free algorithm eliminates that loop
This package provides a simple way to enable basic seccomp system call filtering in any application (even proprietary one) via environment variables.
Ck_ec implements 32- and 64- bit event counts. Event counts let us easily integrate OS-level blocking (e.g., futexes) in lock-free protocols. Waiters block conditionally, if the event count's value is still equal to some old value.
Gcassert is a program for making assertions about compiler decisions in Golang programs, via inline comment directives like //gcassert:inline
We present a general transformation that takes any concurrent data structure written using CAS and adds wait-free linearizable query operations to it. These query operations may access arbitrary parts of the data structure, and do not interfere with the progress or running time of other operations. For example, our transformation can be used to add efficient and linearizable range queries, predecessor queries, and top-k queries to existing concurrent set data structures. We achieve this by presenting an efficient technique for taking lazy snapshots of CAS-based data structures.
Big Data is defined as high volume of variety of data with an exponential data growth rate. Data are amalgamated to generate revenue, which results a large data silo. Data are the oils of modern IT industries. Therefore, the data are growing at an exponential pace. The access mechanism of these data silos are defined by metadata. The metadata are decoupled from data server for various beneficial reasons. For instance, ease of maintenance. The metadata are stored in metadata server (MDS). Therefore, the study on the MDS is mandatory in designing of a large scale storage system. The MDS requires many parameters to augment with its architecture. The architecture of MDS depends on the demand of the storage system's requirements. Thus, MDS is categorized in various ways depending on the underlying architecture and design methodology. The article surveys on the various kinds of MDS architecture, designs, and methodologies. This article emphasizes on clustered MDS (cMDS) and the reports are prepared based on a) Bloom filter−based MDS, b) Client−funded MDS, c) Geo−aware MDS, d) Cache−aware MDS, e) Load−aware MDS, f) Hash−based MDS, and g) Tree−based MDS. Additionally, the article presents the issues and challenges of MDS for mammoth sized data.
Modern computing systems are highly concurrent. Threads run concurrently in shared-memorymulti-core systems, and programs run in different servers communicating by sending messages to eachother. Concurrent programming is hard because it requires to cope with many possible, unpredictablebehaviors of the processes, and the communication media. The article argues that right from the startin 1960’s, the main way of dealing with concurrency has been by reduction to sequential reasoning. Ittraces this history, and illustrates it through several examples, from early ideas based on mutual exclu-sion (which was initially introduced to access shared physical resources), passing through consensus andconcurrent objects (which are immaterial data), until today distributed ledgers. A discussion is also pre-sented, which addresses the limits that this approach encounters, related to fault-tolerance, performance,and inherently concurrent problems
In this article, we focus on latencies rather than throughput. We first document the fact that LSM KVs exhibit high tail latencies. The techniques that have been proposed for optimizing throughput do not address this issue, and, in fact, in some cases, exacerbate it. The root cause of these high tail latencies is interference between client writes, flushes, and compactions. Another major cause for tail latency is the heterogeneous nature of the workloads in terms of operation mix and item sizes whereby a few more computationally heavy requests slow down the vast majority of smaller requests. We introduce the notion of an Input/Output (I/O) bandwidth scheduler for an LSM-based KV store to reduce tail latency caused by interference of flushing and compactions and by workload heterogeneity. We explore three techniques as part of this I/O scheduler: (1) opportunistically allocating more bandwidth to internal operations during periods of low load, (2) prioritizing flushes and compactions at the lower levels of the tree, and (3) separating client requests by size and by data access path. SILK+ is a new open-source LSM KV that incorporates this notion of an I/O scheduler.
The authors proposed four desirable properties in transaction processing systems forachieving low-latency ofreadtransactions, with asynchronous and reliable communications, andreferred to them collectively as theSNOW properties: The underlying properties, in the contextof an execution, are (i)strict serializability(S) property where read andwritetransactions seemto occur atomically; (ii)non-blocking(N) property implies that for every read operation on anyobject, during areadtransaction, the response at the corresponding server is non-blocking; (iii)one version and one round(O) property implies every read operation, during a read transaction,completes inone-roundof client-server communication and the respective server responds withonly one version of the object value; and (iv)concurrentwritetransactions(W) property statesthatreadtransactions can have concurrentwritetransactions. Then they argued that it isimpossible to implement all the four properties, in the same system, even with at least threeclients. They referred to their result as theSNOWtheorem, and they posed the two-clientsetting as an open question.
The exponential growth of digital information is imposing increasing scale and efficiency demands on mod-ern storage infrastructures. As infrastructure complexity increases, so does the difficulty in ensuring qualityof service, maintainability, and resource fairness, raising unprecedented performance, scalability, and pro-grammability challenges. Software-Defined Storage (SDS) addresses these challenges by cleanly disentanglingcontrol and data flows, easing management, and improving control functionality of conventional storage sys-tems. Despite its momentum in the research community, many aspects of the paradigm are still unclear, un-defined, and unexplored, leading to misunderstandings that hamper the research and development of novelSDS technologies. In this article, we present an in-depth study of SDS systems, providing a thorough descrip-tion and categorization of each plane of functionality. Further, we propose a taxonomy and classification ofexisting SDS solutions according to different criteria. Finally, we provide key insights about the paradigm anddiscuss potential future research directions for the field.
This guide walks through creating PlantUML diagrams
In today's enterprise storage systems, supported data services such as snapshot delete or drive rebuild can cause tremendous performance interference if executed inline along with heavy foreground IO, often leading to missing SLOs (Service Level Objectives). Typical storage system applications such as web or VDI (Virtual Desktop Infrastructure) follow a repetitive high/low workload pattern that can be learned and forecasted. We propose a priority-based background scheduler that learns this repetitive pattern and allows storage systems to maintain peak performance and in turn meet service level objectives (SLOs) while supporting a number of data services. When foreground IO demand intensifies, system resources are dedicated to service foreground IO requests and any background processing that can be deferred are recorded to be processed in future idle cycles as long as forecast shows that storage pool has remaining capacity. The smart background scheduler adopts a resource partitioning model that allows both foreground and background IO to execute together as long as foreground IOs are not impacted where the scheduler harness any free cycle to clear background debt. Using traces from VDI application, we show how our technique surpasses a method that statically limit the deferred background debt and improve SLO violations from 54.6% when using a fixed background debt watermark to merely a 6.2% if dynamically set by our smart background scheduler.
Cloud providers offer a variety of execution platforms in form of bare-metal, VM, and containers. However, due to the pros and cons of each execution platform, choosing the appropriate platform for a specific cloud-based application has become a challenge for solution architects. The possibility to combine these platforms (e.g. deploying containers within VMs) offers new capacities that makes the challenge even further complicated. However, there is a little study in the literature on the pros and cons of deploying different application types on various execution platforms. In particular, evaluation of diverse hardware configurations and different CPU provisioning methods, such as CPU pinning, have not been sufficiently studied in the literature. In this work, the performance overhead of container, VM, and bare-metal execution platforms are measured and analyzed for four categories of real-world applications, namely video processing, parallel processing (MPI), web processing, and No-SQL, respectively representing CPU intensive, parallel processing, and two IO intensive processes. Our analyses reveal a set of interesting and sometimes counterintuitive findings that can be used as best practices by the solution architects to efficiently deploy cloud-based applications. Here are some notable mentions: (A) Under specific circumstances, containers can impose a higher overhead than VMs; (B) Containers on top of VMs can mitigate the overhead of VMs for certain applications; (C) Containers with a large number of cores impose a lower overhead than those with a few cores
The efficient implementation of function calls and non-local control transfers is a critical part of modern language implementations and is important in the implementation of everything from recursion, higher-order functions, concurrency and coroutines, to task-based parallelism. In a compiler, these features can be supported by a variety of mechanisms, including call stacks, segmented stacks, and heap-allocated continuation closures. An implementor of a high-level language with advanced control features might ask the question "what is the best choice for my implementation?" Unfortunately, the current literature does not provide much guidance, since previous studies suffer from various flaws in methodology and are outdated for modern hardware. In the absence of recent, well-normalized measurements and a holistic overview of their implementation specifics, the path of least resistance when choosing a strategy is to trust folklore, but the folklore is also suspect. This paper attempts to remedy this situation by providing an "apples-to-apples" comparison of six different approaches to implementing call stacks and continuations. This comparison uses the same source language, compiler pipeline, LLVM-backend, and runtime system, with the only differences being those required by the differences in implementation strategy. We compare the implementation challenges of the different approaches, their sequential performance, and their suitability to support advanced control mechanisms, including supporting heavily threaded code. In addition to the comparison of implementation strategies, the paper's contributions also include a number of useful implementation techniques that we discovered along the way.
Smoltcp is a standalone, event-driven TCP/IP stack that is designed for bare-metal, real-time systems. Its design goals are simplicity and robustness. Its design anti-goals include complicated compile-time computations, such as macro or type tricks, even at cost of performance degradation
The libpoireau library intercepts a small fraction of calls to malloc/calloc/etc., to generate a statistically representative overview of an application's heap footprint
In this paper, we set out the goal to revisit the results of “Starringinto the Abyss [...] of Concurrency Control with [1000] Cores” [27]and analyse in-memory DBMSs on today’s large hardware. Despitethe original assumption of the authors, today we do not see single-socket CPUs with 1000 cores. Instead multi-socket hardware madeits way into production data centres. Hence, we follow up on thisprior work with an evaluation of the characteristics of concurrencycontrol schemes on real production multi-socket hardware with1568 cores. To our surprise, we made several interesting findingswhich we report on in this paper.
Hash tables are an essential data-structure for numerous networking applications (e.g., connection tracking, firewalls, network address translators). Among these, cuckoo hash tables provide excellent performance by allowing lookups to be processed with very few memory accesses (2 to 3 per lookup). Yet, for large tables, cuckoo hash tables remain memory bound and each memory access impacts performance. In this paper, we propose algorithmic improvements to cuckoo hash tables allowing to eliminate some unnecessary memory accesses; these changes are conducted without altering the properties of the original cuckoo hash table so that all existing theoretical analysis remain applicable. On a single core, our hash table achieves 37M lookups per second for positive lookups (i.e., when the key looked up is present in the table), and 60M lookups per second for negative lookups, a 50% improvement over the implementation included into the DPDK. On a 18-core, with mostly positive lookups, our implementation achieves 496M lookups per second, a 45% improvement over DPDK.
The fundamental challenges that every researcher, systems architect, or developer faces when designing a new access method are how to minimize, i) read times (R), ii) update cost (U), and iii) memory (or storage) overhead (M). In this project we first conjecture that when optimizing the read-update-memory overheads, optimizing in any two areas negatively impacts the third.
A collection of tools for developers who have little to no artistic talent.
Htmx allows you to access AJAX, WebSockets and Server Sent Events directly in HTML, using attributes, so you can build modern user interfaces with the simplicity and power of hypertext
DuckDB is an embedded database designed to execute analytical SQL queries fast while embedded in another process
C library for accessing the PostgreSQL parser outside of the server. This library uses the actual PostgreSQL server source to parse SQL queries and return the internal PostgreSQL parse tree.
"Distroless" images contain only your application and its runtime dependencies. They do not contain package managers, shells or any other programs you would expect to find in a standard Linux distribution.
For decades, applications deployed on a world-wide scale have been forced to give up at least one of (1) strict serializability (2) low latency writes (3) high transactional throughput. In this paper we discuss SLOG: a system that avoids this tradeoff for workloads which contain physical region locality in data access. SLOG achieves high-throughput, strictly serializable ACID transactions at geo-replicated distance and scale for all transactions submitted across the world, all the while achieving low latency for transactions that initiate from a location close to the home region for data they access. Experiments find that SLOG can reduce latency by more than an order of magnitude relative to state-of-the-art strictly serializable geo-replicated database systems such as Spanner and Calvin, while maintaining high throughput under contention.
The shared log paradigm is at the heart of modern distributed applications in the growing cloud computing industry. Often, application logs must be stored durably for analytics, regulations, or failure recovery, and their smooth operation depends closely on how the log is implemented. Scalog is a new implementation of the shared log abstraction that offers an unprecedented combination of features for continuous smooth delivery of service: Scalog allows applications to customize data placement, supports reconfiguration with no loss in availability, and recovers quickly from failures. At the same time, Scalog provides high throughput and total order.
Llvm-mca is a performance analysis tool that uses information available in LLVM (e.g. scheduling models) to statically measure the performance of machine code in a specific CPU. Performance is measured in terms of throughput as well as processor resource consumption. The tool currently works for processors with an out-of-order backend, for which there is a scheduling model available in LLVM.
An error kernel is the part of an application or system that should never fail. This is ideally as small as possible since the less surface area that it has, the less likely catastrophic failures will happen. When they do happen, the core system is small enough to reason about which greatly helps with triage and cauterizing of wounds.
C++ reactors, coros, datastructures, etc
Today I want to criticize the whole logging culture and provide a bunch of alternatives.
This is a collection of lectures and labs Linux kernel topics. The lectures focus on theoretical and Linux kernel exploration. The labs focus on device drivers topics and they resemble “howto” style documentation
MeiliSearch is a RESTful search API. It aims to be a ready-to-go solution for everyone who wants a fast and relevant search experience for their end-users
In this document, the internals of PostgreSQL for database administrators and system developers are described.
Quicly is a QUIC implementation, written from the ground up to be used within the H2O HTTP server.
Quiche is an implementation of the QUIC transport protocol and HTTP/3 as specified by the IETF. It provides a low level API for processing QUIC packets and handling connection state. The application is responsible for providing I/O (e.g. sockets handling) as well as an event loop with support for timers.
Random replication is widely used in data center storage systems to prevent data loss. However, random replication is almost guaranteed to lose data in the common scenario of simultaneous node failures due to cluster-wide power outages. Due to the high fixed cost of each incident of data loss, many data center operators prefer to minimize the frequency of such events at the expense of losing more data in each event. We present Copyset Replication, a novel general-purpose replication technique that significantly reduces the frequency of data loss events
This article discusses a new replica placement technique that reduces the probability of data loss.
I am a strong believer that Bourne-derived languages are extremely bad, on the same order of badness as Perl, for programming, and consider programming sh for any purpose other than as a super-portable, lowest-common-denominator platform for build or bootstrap scripts and the like, as an extremely misguided endeavor.
Lightweight, fault-tolerant message streams
Library for Restartable Sequences.
Data-oriented design is inspired by high-performance computing techniques, database design, and functional programming values. It provides a practical methodology that reduces complexity while improving performance of both your development team and your product. Understand the goal, understand the data, understand the hardware, develop the solution
Tracking, Benchmarking and Sharing Information about an open source embedded data storage engines, internals, architectures, data storage and transaction processing.
MsQuic is a Microsoft implementation of the IETF QUIC protocol. It is cross platform, written in C and designed to be a general purpose QUIC library
NPF is a layer 3 packet filter, supporting stateful packet inspection, IPv6, NAT, IP sets, extensions and many more. It uses BPF as its core engine and it was designed with a focus on high performance, scalability, multi-threading and modularity. NPF was written from scratch in 2009. It is written in C99 and distributed under the 2-clause BSD license. NPF is provided as a userspace library to be used in a bespoke application to process packets. It can run on Linux, typically, in combination with such frameworks like Data Plane Development Kit (DPDK) or netmap.
This informational document is an attempt to make the case for implementing network protocols without performing I/O, and to provide concrete assistance and instructions for doing so in Python
A small and efficient runtime for WebAssembly & WASI
WAVM is a WebAssembly virtual machine, designed for use in non-web applications.
Lucet is a native WebAssembly compiler and runtime. It is designed to safely execute untrusted WebAssembly programs inside your application
Krustlet acts as a Kubelet by listening on the event stream for new pods that the scheduler assigns to it based on specific Kubernetes tolerations. The default implementation of Krustlet listens for the architecture wasm32-wasi and schedules those workloads to run in a wasmtime-based runtime instead of a container runtime
I’m writing the book in Markdown, then use Pandoc to convert it to PDF and EPUB.
Setting up a new C++ project usually requires a significant amount of preparation and boilerplate code, even more so for modern C++ projects with tests, executables and contiguous integration. This template is the result of learnings from many previous projects and should help reduce the work required to setup up a modern C++ project.
Pg_auto_failover is an extension and service for PostgreSQL that monitors and manages automated failover for a Postgres cluster. It is optimized for simplicity and correctness and supports Postgres 10 and newer
The notion of consistency is used across different computer science disciplines from distributed systems to database systems to computer architecture. It turns out that consistency can mean quite different things across these disciplines, depending on who uses it and in what context it appears. We identify two broad types of consistency, state consistency and operation consistency, which differ fundamentally in meaning and scope. We explain how these types map to the many examples
This is a list of resources related to LD_PRELOAD, a mechanism for changing application behavior at run-time. Libraries can override specified functions with another, for example, making time(3) always return 0. This is often useful for testing or modifying application behavior without source code changes.
Distributed consensus is a fundamental primitive for constructing fault-tolerant, strongly-consistent distributed systems. Though many distributed consensus algorithms have been proposed, just two dominate production systems: Paxos, the traditional, famously subtle, algorithm; and Raft, a more recent algorithm positioned as a more understandable alternative to Paxos. In this paper, we consider the question of which algorithm, Paxos or Raft, is the better solution to distributed consensus? We analyse both to determine exactly how they differ by describing a simplified Paxos algorithm using Raft's terminology and pragmatic abstractions. We find that both Paxos and Raft take a very similar approach to distributed consensus, differing only in their approach to leader election. Most notably, Raft only allows servers with up-to-date logs to become leaders, whereas Paxos allows any server to be leader provided it then updates its log to ensure it is up-to-date. Raft's approach is surprisingly efficient given its simplicity as, unlike Paxos, it does not require log entries to be exchanged during leader election. We surmise that much of the understandability of Raft comes from the paper's clear presentation rather than being fundamental to the underlying algorithm being presented.
It is a reliable replication protocol inspired by the multiprocessor's cache-coherence protocols. Hermes combines a broadcast-based, invalidating design with logical timestamps and the idea of early value propagation to achieve the following desired properties for reliable datastores.