Performance Zone is brought to you in partnership with:

I am a software architect working in service hosting area. I am interested and specialized in SaaS, Cloud computing and Parallel processing. Ricky is a DZone MVB and is not an employee of DZone and has posted 88 posts at DZone. You can read more from them at their website. View Full User Profile

Design for Scalability

  • submit to reddit

Building scalable system is becoming a hotter and hotter topic. Mainly because more and more people are using computer these days, both the transaction volume and their performance expectation has grown tremendously.


General Principles


"Scalability" is not equivalent to "Raw Performance"

  • Scalability is about reducing the adverse impact due to growth on performance, cost, maintainability and many other aspects
  • e.g. Running every components in one box will have higher performance when the load is small. But it is not scalable because performance drops drastically when the load is increased beyond the machine's capacity

Understand environmental workload conditions that the system is design for

  • Dimension of growth and growth rate: e.g. Number of users, Transaction volume, Data volume
  • Measurement and their target: e.g. Response time, Throughput

Understand who is your priority customers

  • Rank the importance of traffic so you know what to sacrifice in case you cannot handle all of them

Scale out and Not scale up

  • Scale the system horizontally (adding more cheap machine), but not vertically (upgrade to a more powerful machine)

Keep your code modular and simple

  • The ability to swap out old code and replace with new code without worries of breaking other parts of the system allows you to experiment different ways of optimization quickly
  • Never sacrifice code modularity for any (including performance-related) reasons

Don't guess the bottleneck, Measure it

  • Bottlenecks are slow code which are frequently executed. Don't optimize slow code if they are rarely executed
  • Write performance unit test so you can collect fine grain performance data at the component level
  • Setup a performance lab so you can conduct end-to-end performance improvement measurement easily

Plan for growth

  • Do regular capacity planning. Collect usage statistics, predict the growth rate

Common Techniques

Parallel Processing

  • Split the work into smaller chunks and assign each chunk to a pool of worker machines. When the size of work grows, you just need to add more workers into the pool (grow the pool but not individual workers) and your cost grow linearly with respect to the work load.
  • However, this requires the work (algorithm) itself to be parallelizable. This usually mean the steps of execution should be independent of each other.
  • Google's Map/Reduce is a good framework for this model. There is also an open source Java framework Hadoop as well.

Reuse Previous Result

  • This is a time vs space tradeoff. Some executions may use the same set of input parameters over and over again. Therefore, instead of redo the same execution for same input parameters, we can remember the previous execution's result.
  • This is typically implemented as a lookup cache.
  • Memcached and EHCache are some of the popular caching packages

Asynchronous Processing

  • You make a call which returns a result. But you don't need to use the result until at a much later stage of your process. Therefore, you don't need to wait immediately after making the call., instead you can proceed to do other things until you reach the point where you need to use the result.
  • In additional, the waiting thread is idle but consume system resources. For high transaction volume, the number of idle threads is (arrival_rate * processing_time) which can be a very big number if the arrival_rate is high. The system is running under a very ineffective mode
  • The service call in this example is better handled using an asynchronous processing model. This is typically done in 2 ways: Callback and Polling
  • In callback mode, the caller need to provide a response handler when making the call. The call itself will return immediately before the actually work is done at the server side. When the work is done later, response will be coming back as a separate thread which will execute the previous registered response handler. Some kind of co-ordination may be required between the calling thread and the callback thread.
  • In polling mode, the call itself will return a "future" handle immediately. The caller can go off doing other things and later poll the "future" handle to see if the response if ready. In this model, there is no extra thread being created so no extra thread co-ordination is needed.

Implementation design considerations

  • Use efficient algorithms and data structure. Analyze the time (CPU) and space (memory) complexity for logic that are execute frequently (ie: hot spots). For example, carefully decide if hash table or binary tree should be use for lookup.
  • Analyze your concurrent access scenarios when multiple threads accessing shared data. Carefully analyze the synchronization scenario and make sure the locking is fine-grain enough. Also watch for any possibility of deadlock situation and how you detect or prevent them. A wrong concurrent access model can have huge impact in your system's scalability. Also consider using Lock-Free data structure (e.g. Java's Concurrent Package have a couple of them)
  • Analyze the memory usage patterns in your logic. Determine where new objects are created and where they are eligible for garbage collection. Be aware of the creation of a lot of short-lived temporary objects as they will put a high load on the Garbage Collector.
  • However, never trade off code readability for performance. (e.g. Don't try to bundle too much logic into a single method). Let the VM handle this execution for you.
Published at DZone with permission of Ricky Ho, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)


Torbjörn Gannholm replied on Wed, 2008/05/28 - 3:31pm

Have you measured the impact of short-lived temporary objects? I haven't studied the gc impact in depth, but according to theory and the few measurements I have performed, short-lived objects are almost free. In particular, creating a new immutable object and garbage collecting it is generally much cheaper than performing synchronized calls to a mutable object.

Dmitriy Setrakyan replied on Wed, 2008/05/28 - 4:31pm

Google's Map/Reduce is a good framework for this model. There is also an open source Java framework Hadoop as well.


There is also another open source Java framework for parallel processing - GridGain.

Dmitriy Setrakyan
GridGain - Grid Computing Made Simple

Dmitriy Setrakyan replied on Wed, 2008/05/28 - 4:34pm

However, never trade off code readability for performance.

Couldn't agree more.

Dmitriy Setrakyan
GridGain - Grid Computing Made Simple

Ricky Ho replied on Wed, 2008/05/28 - 9:32pm in response to: Torbjörn Gannholm

Short-lived temporary objects will increase the load of GC.  But in most case, people are not create temporary object to avoid "synchronized", they just create temp object in string manipulation or just create a function object, which can be eliminated easily.

 Inappropriate "synchronized" is actually causing a bigger scalability problem than temp object.  Use AtomicXXX if you can, and use the LOCK-Free data structure from Java concurrent package to avoid "synchronized" as much as possible.

Rgds, Ricky

Rahul Hinge replied on Fri, 2009/04/17 - 1:24pm

Hi, How can we ensure the transaction scalability at design time. For example, Customer is expecting system to support 100000 transactions per minute. If the developed and deployed system proved to be unable to handle 100000 trx/minute then there is high penalty clause. I don't have a luxury to create actual prototype or POC. In this case, how should I ensure transaction scalability and satisfy customer? Thanks and Regards, Rahul

jame jack replied on Wed, 2009/06/17 - 7:39pm

تحميل برامج برامج جوالات العاب بنات تكنولوجيا كتب تعليم UltraSurf Internet Download Manager ProgDVB برامج مجانية أفضل المواقع العربية مشاهدة محطات مشفرة Online TV Player Internet Download Manager 5.17 Build 4 رقص شرقي anyTV Pro 4.32 OnLineLive 7.1.1 هزي يانواعم ProgDVB 6.06.2 SopCast 3.0.3 Falco Image Studio 3.6 لعبة تزلج على الجليد UltraSurf 9.4 كاثرين هيغل Katherine Heigl محطة غنوة FreeZ Online TV 1.0 Free Video to Mp3 Converter Advanced MP3 Converter 2.10 Xilisoft Video to Audio Converter Blaze Media Pro 8.02 AKRAM Media Creator 1.11 DVD Audio Extractor 4.5.4 Free WMA to MP3 Converter 1.16 لعبة نينجا المتقدم لعبة قذف كرة لعبة دراجات البهلوانية لعبة اعداء الغابة تحميل برامج Download DivX Subtitles 2.0 BullGuard 8.5 Google Chrome Dev Dell Studio XPS Desktop 435T Intel Matrix Storage Manager A00 Gigabyte GA-EP45-UD3P Bios F9 Ambush HDConvertToX MSI Wind Nettop CS 120 Realtek Audio Driver Biostar T41-A7 6.x Realtek On-Board Audio Driver for 2000/2003/XP TweakNow RegCleaner 4.1.1 SpeedItup Free 4.97 برامج العاب - Internet Download Manager - برامج جوالات - العاب - محطة غنوة - قنوات فضائية - بنات - تكنولوجيا - كتب تعليم - UltraSurf -

Ming Shing replied on Thu, 2010/11/25 - 4:30am

Your blog is very much useful and rocking dear. you have provided detailed about Dvd's today's latest technologies. If you want to get the knowledge of latest technologies, then you must take a latest information. For the some developing success, you can get information about mcdba certification. pass4sure MB5-629, pass4sure MB5-858, pass4sure MB7-842 Thanks a lot for your great post. I really enjoy your post and also wait for your latest posts. Well done.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.