Enterprise Integration Zone is brought to you in partnership with:

Steven is CEO and Co-founder of 3scale networks: one of the worlds leading providers of API Management Infrastructure (http://www.3scale.net). Steven is a DZone MVB and is not an employee of DZone and has posted 63 posts at DZone. You can read more from them at their website. View Full User Profile

Welcome to Distributed

05.27.2013
| 2589 views |
  • submit to reddit
This year’s Gluecon as always was a mix ideas and memes (a great event as always!). Reflecting on the talks and conversations on the flight back, there seemed to be one to us which stood our more than any other, which was nicely captured by Michael Rose‘s Tweet:

In fact, we always were, but conveniently forget that fact most of the time. Specifically, APIs, Cloud deployments, SAAS and “remote access to everything” make building systems incredibly hard because they are distributedacross organisations. The fact that building systems in which multiple components that we depend on are hosted in remote infrastructure owned and operated by others has been a reality for a good long while. However, the increasing adoption of APIs in particular, simply makes the issues that arise more acute – and we don’t talk about the enough. The theme came out strongly both in John Sheehan‘s Keynote on Runscope:

and in Andy Gross‘s:

John’s talk covered the pain of working with multiple different external APIs – both at development time and at run-time. Each API fails in different ways and at different times – making even gaining insights as to what might be wrong hard to glean, particular with systems that need to run 24/7. Andy dug right in on the kind of testing needed to really validate that a distributed system such as the deployments Basho – “Concurrency is no longer enough – We are distributed Systems people now”.

The theme also came out in Talks from Netflix, Google and Parse – not only do internal systems (which may be distributed) need to be managed closely: there are now external systems which depend on these and others on these.

In a “distributed” world many things become harder: remote nodes cannot always be relied upon, clocks are not synchronized, small latencies between locations make many algorithms invalid or impossibly slow, many of the endpoints are managed by different organisations – meaning changes are often not coordinated or orchestrated.

All this warms ours hearts as an “old time” distributed systems folks – Internet Applications are becoming genuinely distributed across organisations. This creates growing pains – but also makes new types of application possible. People often ask “what’s the fundamental difference between SOA and APIs? The essence of the answer is, not technical – it is organisational:

APIs are typically designed for inter-organizational integration, SOA typically for intra-organizational.

Even though SOA solutions do get used for the former, this has almost always happened with strong assumptions on the human collaboration around the integration and sense of “shared ownership” of the resulting app – control of the resources involved. With the rapidly increasing number of API integration, this assumption is simply not there. Apps in an API world need to handle being truly distributed.

Vendors like ourselves and new companies such as Runscope, will hopefully start to make it easier for both sides of the API equation to take some of the pain out of providing and consuming APIs. Building genuinely functioning, stable apps across clouds is very hard and we have a long way to go – but we’re starting to recognize and address the pain points!

Huge congrats to Eric, Kim and the team for another great event!

Published at DZone with permission of Steven Willmott, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Tags: