Mitch Pronschinske is a Senior Content Analyst at DZone. That means he writes and searches for the finest developer content in the land so that you don't have to. He often eats peanut butter and bananas, likes to make his own ringtones, enjoys card and board games, and is married to an underwear model. Mitch is a DZone Zone Leader and has posted 2573 posts at DZone. You can read more from them at their website. View Full User Profile

Redis 2 and "Ridiculous Speed Improvements"

05.17.2010
| 5191 views |
  • submit to reddit
Having been recently hired by VMware, Redis creator Salvatore Sanfilippo, is getting more time and resources to work on his open source NoSQL database.  As the first release candidate for Redis 2.0 nears, Sanfilippo is finishing some features that are sure to make users very happy.  New features include significant performance optimizations, a new test suite, and an AOF check tool.

Redis is a speedy key-value store with a rich API.  It's similar to memcached because values can be strings, but the dataset is not volatile.  Unlike memcached, Redis' values can also be lists, sets, hashmaps, blobs, and ordered sets.  It also allows you to append a value to the end of an item list that has already been stored for a given key.  All of its data types can be manipulated with atomic operations to push/pop elements, add/remove elements, perform server side union, intersection, difference between sets, and more.

Optimization
Before this upcoming release, Sanfilippo hadn't gotten the chance to check the performance of individual pieces of code.  Redis was already fast at the time, so optimization was not high on his list of priorities.  Using a few test datasets from production environments, Sanfilippo started optimizing different parts of the code and got the following results:
1.2.6 loading time: 21 seconds
1.2.6 saving time: 7.5 seconds

1.3.12 loading time: 3 seconds
1.3.12 saving time: 1.2 seconds
These times are from the same dataset (cached in the OS buffers).  In real world situations where the I/O is also involved, the performance boost will not always be as high, but it's still a big improvement.  Sanfilippo says the speed increase alone is worth the upgrade.  

AOF Tool
Pieter Noordhuis contributed the redis-check-aof tool, which is able to check the AOF (Aggregate Objective Function) for consistency and then repair it by truncating the last non complete command if needed.  The tool is a life saver when a large, corrupted AOF file needs to be truncated at the end quickly so the server can be restarted faster.  Sanfilippo plans to optimize AOF in Redis 2.2, but he suspects it is already running much faster because of the general optimizations he made.  

Testing Suite
This feature was also contributed by Noordhuis.  The suite is able to run a dedicated Redis server and multiple servers can be started with various configurations.  This means that you can now test for replication, auto command, and more.  The new DEBUG DIGEST command, which can take the SHA1 of the whole dataset, is now being used to check the consistency of RDB/AOF files.  New integration tests provide a way for this command to automatically check replication consistency.  Support for valgrind is also being added currently.

Other new features coming in Redis 2.0 include VM optimization and Pub/Sub matching support.  A full list of features will be available on the release date.  In version 2.2, Sanfilippo says he wants to work on increasing Redis' reliability, enhancing documentation, and reducing memory footprint.  There are several features he said that he would like to add, such as stronger string type controls, MULTI/EXEC with CAS support, and a good clustering solution, but right now he's slowing down the process of adding new things.

Redis 2.0 RC1 should be available on May 21st 2010.