Build a new runtime platform for Redis customers

Over the past decade, Say it again has become a popular and widely used in-memory cache.

unlike most distributed cachesit offers a compelling set of built-in data structures that enhance its capabilities beyond just storing serialized objects. Created in 2009 As a single-server store to help design a web server, Redis gives applications many useful options for organizing and retrieving stored data, including sets, lists, hashes, and sorted sets.

A few years later he added cluster support to scale throughput and add data replication for high availability. However, the cluster implementation of Redis introduced specialized concepts, such as hashslots and replica shards, that system administrators needed to understand and manage.

Along with Redis’ use of eventual rather than total coherencethese concepts have created complexity that makes cluster management more difficult while reducing flexibility and reliability.

On the other hand, ScaleOut StateServer®an enterprise-focused distributed cache for serialized objects first released in 2005was designed from the ground up to run on a cluster of servers with the specific design goals of high performance and reliable data storage while maximizing ease of use.

It automates load balancing, replication and data recovery and stores data with full consistency (i.e. sequential consistency) on replicas. It also executes client requests using all available processing cores for maximum throughput.

These features greatly simplify cluster management, improve flexibility, and reduce total cost of ownership, especially for enterprise users. For example, unlike Redis, ScaleOut server clusters can scale seamlessly from a single server to multiple servers, and system administrators don’t need to manage hashslots or shard replicas.

The challenge – integrating Redis command processing into a new clustering architecture

Scaling software recognized that running Redis commands on a ScaleOut StateServer cluster would give users the best of both worlds: familiar, rich data structures combined with much simpler cluster management and full data consistency.

However, the ideal implementation should use actual open source Redis code to execute commands so that client commands behave the same as open source Redis clusters. The challenge then was to integrate the Redis code into the ScaleOut StateServer runtime platform and take advantage of its highly automated clustering capabilities while eliminating the single-threaded constraints of Redis. event loop architecture.

The company recently made this integration, and in ScaleOut StateServer version 5.11, posted a community preview which introduces support for the most common Redis data structures (strings, sets, lists, hashes, and sorted sets) as well as publish/subscribe commands, transactions, and various utility commands (such as FLUSHDB and EXPIRE) on Windows and Linux. He uses open-source redis Version 6.2.5 code in servers to process Redis commands.

Automated cluster operations, scalable throughput, and full data consistency

With this integration, Redis clients can connect to a ScaleOut StateServer cluster of one or more servers using the standard Redis RESP protocol.

Redis client libraries obtain the hashslot mapping on servers by internally calling the CLUSTER SLOTS or CLUSTER NODES commands, then directing Redis access requests to the appropriate ScaleOut server.

When cluster membership changes, clients automatically use the RESP protocol to redirect commands to the appropriate servers.

To optimize throughput, each ScaleOut server processes incoming Redis commands on multiple threads using all available CPU cores. You don’t need to deploy multiple partitions on each server to scale performance or ensure data replication; these functions are implemented in a process on each server.

As the need for additional throughput grows, system administrators can simply join new servers to the cluster. ScaleOut StateServer automatically rebalances hashslots in the cluster as servers are added or removed. It also delays execution of Redis commands during load balancing (and recovery) to maintain a consistent picture of hashslot placement and avoid client exceptions. After a hashslot is fully migrated to a remote server, client requests return a -MOVED indication so that they can be redirected to the new server.

Additionally, ScaleOut StateServer automatically creates replicas for all hashslots. There is no need to manually create primary and replica shards or move them from server to server during membership changes and recovery.

ScaleOut StateServer automatically places replica hashslots on different servers from their corresponding primary hashslots and migrates them as needed on membership changes to ensure optimal load balancing. If a server fails or suffers a network outage, ScaleOut StateServer “heals itself” by promoting replicas to primary and creating new replicas as needed.

Redis uses an eventual consistency model to update replicas. Although this model is fast, it is susceptible to providing stale data to clients in the event of a server or network outage. To avoid this problem, ScaleOut StateServer implements fully consistent updates of stored objects using a patented quorum algorithm. When a server receives a Redis command, it completes execution of the command on a quorum containing the main hashslot and its replicas before returning to the client. Transactions are processed the same way.

In summary – new features for Redis users

ScaleOut Software’s innovative integration of open source Redis code into its cluster architecture will bring powerful new features to Redis users while ensuring native Redis behavior for client applications.

Enterprise users now have an easier way to manage Redis clusters with automated cluster management and built-in throughput scaling, and they benefit from fully consistent updates to stored data performed by Redis commands.

They can also access native ScaleOut StateServer APIs, which incorporate advanced features not available on open-source Redis clusters, including data-parallel computing, streaming analytics, and consistent and extensive data replication.

Want to learn more about cybersecurity and the cloud from industry leaders? To verify Cybersecurity and Cloud Expo taking place in Amsterdam, California and London.

Check out other upcoming enterprise technology events and webinars powered by TechForge here.

Keywords: redis, scaling software

About Jon Moses

Check Also

Intel promises “substantial contributions” to the growth of RISC-V • The Register

Analysis Here’s something that would have seemed odd just a few years ago: to help …