Enterprise Integration Zone is brought to you in partnership with:

Ross founded the open source Mule® project in 2003. Frustrated by integration "donkey work," he set out to create a new platform that emphasized ease of development and re-use of components. He started the Mule project to bring a modern approach, one of assembly, rather than repetitive coding, to developers worldwide. He is now the Founder and CTO of Mulesoft. Ross is a DZone MVB and is not an employee of DZone and has posted 90 posts at DZone. You can read more from them at their website. View Full User Profile

More data to your Mule with Riak!

07.24.2013
| 2411 views |
  • submit to reddit
Originally authored by David Dossot

In the new enterprise, the “one big database” paradigm is being progressively eroded as it becomes more apparent that the ACID qualities of traditional SQL databases are not always needed, and can actually get in the way of massive scalability. In order to respond to the imperative of cloud deployments, new data stores have emerged.

One of them is Riak, a highly-available, fault-tolerant and scalable distributed key-value store built by Basho Technologies, Inc. on the design principles laid out in the famous Amazon Dynamo paper. One of the key features of Riak is that it “embraces failure” in the sense that, instead of sweeping under the carpet the problematic scenarios (like net-splits), it provides developers with the tools needed to deal with these issues as part of the normal operation mode instead of a leaving them to a (seldom addressed) failure mode. As such, data in Riak is said to be “eventually consistent” because it is only when all the pending replications have been done and the related conflicts have been resolved that data is consistent.

The new Riak Connector for Mule allows you to interact with this data store from your Mule applications, opening the door to new integration scenarios. Let’s review an example of such a scenario.

For this example, we will use the proverbial shopping cart scenario. It is proverbial because it’s the scenario for which Amazon Dynamo was originally built. Therefore it fits Riak perfectly because:

  • It doesn’t have the need for strong data consistency: a customer has the chance to review her cart before validating the order. Discrepancies could surely be annoying but, until the order is placed, data for the shopping cart can still be in a state of flux.

  • It is fairly easy to merge different versions of the shopping cart. Indeed, as data gets replicated across a Riak cluster, conflicts can occur where several versions of the same data exist. In the example of a shopping cart, we can resolve such conflicts by doing a union of the different versions, using the highest quantity for items in common.

In our example, Mule will be used to expose an internal cart management API. It will take care of automatically resolving conflicts using the defined policy so end-users of the API will not have to deal with that. It will also take care of serializing and deserializing cart objects to and from JSON. This is illustrated in the following diagram:

In this Mule application, a private flow is dedicated to retrieving shopping carts and deserializing them to their object representations (as illustrated in UML in the above diagram). This flow configuration is shown hereafter:

<flow name="retrieveShoppingCart" processingStrategy="synchronous">
    <riak:fetch bucketName="shoppingCarts" key="#[cartId]"
        resolver-ref="shoppingCartResolver" />

    <transformer ref="jsonToShoppingCart" />
</flow>

As you see, there’s not much there. The bulk of the work is done by the “shoppingCartResolver”, which takes care of implementing the conflict resolution policy we’ve talked about earlier. To reiterate, this policy states that conflicting shopping carts are merged by unioning their items and picking the item with the highest quantity if there are two of the same. We use MEL to write this policy, as shown in the following code sample:
<spring:bean name="shoppingCartResolver"
    class="org.mule.modules.riak.mel.ExpressionConflictResolver"
    init-method="initialise" p:deserializer-ref="jsonToShoppingCart"
    p:serializer-ref="objectToJson">
    <spring:property name="expression">
        <spring:value><![CDATA[
            resolvedCartItems = [:];

            foreach (cart : siblings) {
                foreach (item : cart.items) {
                    existingItem = resolvedCartItems[item.sku];
                    if (existingItem == null || existingItem.quantity < item.quantity) {
                        resolvedCartItems.put(item.sku, item);
                    }
                }
            }

          resolved = siblings[0];
          resolved.items = resolvedCartItems.values();
          resolved;
        ]]></spring:value>
    </spring:property>
</spring:bean>

Because we have configured serializers on the resolver, the conflicting siblings we have to resolve are actually ShoppingCart instances. Without such serializers, our resolvers would have to deal with raw IRiakObject instances, which could be needed if the conflict resolution strategy requires dealing with low level Riak objects, like vector clocks.

You may be thinking that we’re painting a rosy picture here and that there are in fact tons of extra required configuration. But that is not the case! The following configuration snippet shows all the supporting configuration that is necessary to support the above configuration:

<riak:config>
    <riak:http-client-configuration url="http://localhost:8098/riak" />
</riak:config>

<json:json-to-object-transformer name="jsonToShoppingCart"
    returnClass="com.acme.ShoppingCart" />

<json:object-to-json-transformer name="objectToJson" />

As you can see, this is basic connectivity and transformer configuration. It is good to note that the connector supports both Riak’s HTTP and Protobuf transports and also clustered client configuration.

If you’ve been reluctant to bring Riak in your IT landscape for fear of integration issues, now is the time to give the Riak Connector for Mule a try! Enjoy and, as always, your feedback is very appreciated.




Published at DZone with permission of Ross Mason, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)