Enterprise Integration Zone is brought to you in partnership with:

I Started programming in high school and then when I was 16 started coding in C, didn't do much C++ but instead hopped on Java ship. It is complex engineering and art of balancing multiple factors that keeps me warmed up and makes me passionate about what I do. If I want to describe engineering, I would say it is an art and an engineer is an artist. Currently I am a Senior Consultant in Cignex Technologies architecting and leading open source Portal and ECM solutions. Ali has posted 2 posts at DZone. View Full User Profile

How-Tos for implementation of Open Source Portal and ECM Projects

05.26.2010
| 14718 views |
  • submit to reddit

It is always a big challenge when you want to extend your current web presence and manageability of your content produced by your existing legacy applications. There are a variety of different implementations on how to achieve this goal. In this post I will try to list some of the main criteria and constraints and evaluate their impact and order of magnitude on successful execution of such projects.

  1. Available budget and time (this includes current license fees for proprietary solutions)
  2. Volume of new requirements
  3. Non functional requirements
  4. Complexity of current applications
  5. Volume of Legacy Data
  6. Technologies used in the current applications
  7. Size of client organization, on board technical team and existing technical knowledge
  8. Foreseeable vision in the industry and its impact on you

Based on the constraints above and some additional requirements companies decide about their choice of framework or application.

In fact, your budget and window of time is often the main driver of most of the projects. But how can you receive a solution in a tight window with an already limited budget? It makes sense to adopt an already developed framework and build on top of that to cut time and cost. So, how are you supposed to train your team (if you have any) to pick up that framework? The answer is easy; you need creditable system integrators and reliable open source solution providers. Hence, you will have your project delivered and concurrently train the team during implementation.

Alright, let’s say you are pretty convinced about dancing with open source Portal/ECM technologies. Obviously it is not going to end up like an arranged marriage because you should not pay an arm and a leg for it. You are just dating some open source solution providers, just like high school days. The whole idea is to not anchor yourself.

Now the question is how could an open source Portal/ECM solution help you in satisfying your requirements? To be quite frank, open source frameworks may require light to heavy customization in order to satisfy some of your requirements. What makes them quite compelling is that being open source makes it possible to extend and customize them. Indeed, volume of your requirements should be balanced with your time and budget.

Currently there are a variety of different Open Source Portal and CMS frameworks in the market, frameworks built on Python, Java, PHP, C++, ROR, and basically any language or technology out there. Most of these frameworks come with tons of OOTB (Out of the Box) features and cool stuff. However, often most of these freebies must be polished and re-skinned to match your requirements. In fact, assessing these frameworks and matching their OOTB features with the requirement list is the first thing you need to do.

After benchmarking your requirements with available frameworks or having someone to do it, you can shortlist a handful of products for your project. Next step is to categorize the requirements in respect to the shortlisted frameworks as below:

  1. Requirement is satisfied by OOTB feature: The selected framework will contain the code and services.
  2. Requirement will be satisfied by extending the OOTB feature(s): The customizations and extensions should be done in an extensible pluggable way, I will describe after this section.
  3. Requirement is custom and must be developed. This type of requirement could be broken into the following sub categories:
  • Singular requirement: For instance custom Admin, dashboard, contact us page, custom Portlets, unique user attributes and etc. Basically anything that is unique and not repeatable: Such requirements should be implemented as external services or components as much as possible. You do not need to bind the developed components to any frameworks. This is vital for near future extensions. Especially when you consider the trend of technology where services will be hosted on the cloud in near future.
  • Repeatable requirements: For instance email templates, Product Pages, Order entities, Workflows, Search function, etc. Basically anything that is repeatable or could be generalized to be reusable: For such requirements a custom framework must be designed containing generic extensible services configurable for your repeatable requirements. This is imperative especially with big projects where amount of time put in designing and creating such framework pays off in reducing overall production time

After or even before this analysis you should compare the shortlisted frameworks with your current in house expertise. Basically a framework is discarded if it does not match your in house knowledge. Let’s say your team knows PHP like the back of their hand in which case, it would not be wise to pick a Java based framework. But what if you are coming from a big corporate using several technologies, what should be your approach? What if you want to implement a Portal or ECM aggregating legacy services or content from different sources each built on a different technology?

To answer this question it is better to analyze the impact of complexity of current legacy applications and their data on architecture of the solution first.

You may think your new solution must fully replace the existing legacy applications and the current data should be migrated. I would agree if you are using a solution from the eighties or you want to get rid of licensing fees of proprietary products. Otherwise, most of the time it is not necessary or even wrong to fully replace the existing systems. Let me categorize the cases where you need to replace a product:

  1. The current product is using an old technology and you receive no in house or external support for it. In other words, it is impossible to extend, alter, or even maintain it.
  2. The current produc is an expensive proprietary solution and you need to cut cost. 

In other cases it is much better to wire all or some parts of the old legacy applications with the new Portal or ECM solution. It is recommended to do the wiring through Web Service, REST, CMIS or other widely supported interfaces. By the way, this kind of wiring is affordable if the new Portal or ECM is an Open Source one.

Integration of old and new applications through adaptable and extensible interfaces helps you in cutting cost and reducing production time. In a nutshell, my idea is about maximizing utilization of useful components and increasing their reusability by wrapping them with configurable interfaces.

So far we have discussed different types of requirements and techniques to implement them and how to integrate applications together. It is very important to execute the integration via pluggable, extensible and reconfigurable interfaces. Following these design patterns enables you to be able to box solutions in atomic packages and make them independent of each other. All you need to do is to make sure that they can communicate with each other. This way your dependency on technologies will be reduced and your near future customizations could be done much easier.

As mentioned before it is really important what Portal or ECM technology is selected. On the other hand, it is vital to have a flexible Portal and ECM backbone for hosting different types of technologies and content. It is better recognized, if your company is serving a big volume of data produced by different applications. In a nutshell, considering the pace of change in the industry you always need to be able to reuse your existing services instead of renewing them constantly.

You should be hearing cloud computing, cloud services, social networking and things like that every day. The fact is you need to be ready to publish your services to the cloud or be able to consume services from it. In order to do that, you need to focus on the integration level and extension of the current services more than recreating services with new technologies.

As illustrated in the diagram below generic modules and components should be loosely coupled to each other so in future extensions you would go through smaller changes.

 

architecture

 

 

To recap, it is important to be technology agnostic and be flexible to interact with components developed in any language. As depicted in the diagram above it is highly imperative to separate the components from each other and make them as reusable as possible. In order to achieve such goal the following steps are necessary to be considered.

  1. Have the right vision about the future of your requirements.
  2. Prioritize your requirements based on your budget and time and cross check them with the framework you select.
  3. Do not fall in love with any technology, they change every day.
  4. Develop the components in standalone packages and modules. Even avoid developing them within the open source Portal and ECM stack. You do not want to marry them either. Remember, you are just dating them for maximum five years. Standalone packages could later be reconfigured and reused over and over. But if you change the framework chances of reusability drop drastically.
  5. Try to maximize usage of the existing services and save money and time for more important requirements. Try to componentize existing functions and equip them with configurable, extensible and reusable interfaces.
  6. Execute the project by the right people who know the open source stack and technologies inside out. Do not worry to outsource the project. You always can have your people trained on the job during project implementation.
  7. Try to separate rendition of your content from its processing and population. The only thing you need to display in a Portal is the rendered content not the way it is processed. In other words package the rendering layer separately. The view or rendering layer should fetch data from other components through the discussed interfaces.
  8. Do not used heavy technologies in the view layer. Try to use lightweight ones instead. Your Portlets should be designed as shell rendering your already processed content by other modules. If you execute this correctly, you could render your services in any Portal, or application.
  9. Keep your content components separate from the other components and access them through generic and configurable services. You do not need to be bound to one particular CMS framework. Remember you are dating.
  10. The middle layer and business logic services should be template agnostic. The rendering layer is responsible for templating the processed content. The business layer services should be designed in such a way to be consumable by any Portal (for instance iGoogle, iTunes, Facebook, etc.) with minimum effort.

 


Finally, I need to mention that I have exercised this idea for several big clients and it has worked out pretty well. My concept is pretty easy, do not waste your time by repeating yourself, look for already developed code and try to make it reusable and extensible. Ain’t it easy?

Published at DZone with permission of its author, Ali Loghmani.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Daniel Lourenço replied on Thu, 2010/05/27 - 7:32am

Hi Ali, great view on the best practices for the re-architecture of the IT infrastructure for the renewal of the front-end layer while maintain the already existing core IT assets. In fact, I believe that's the great message in re-architecture - understand what has to change, and what should stay the same. People tend to forget the time and effort that is already invested testing, fixing and tunning the existing systems.

One thing that you did not refer was the analysis of the maintainability of the chosen solution - if you are worried about costs, you must analyse your solutions from a maintenance perspective (as we know, most of the software costs are in the maintenance phase, in some situations going up to about 70%). Wouldn't you say?

Analysis of the ease of Change and integration capabilities are also key to guarantee can you can executed a phased modernization process in which you steadily integrate more and more functionality of the existing system in the new one.

Ali Loghmani replied on Fri, 2010/05/28 - 4:51pm

Good point, as a matter of fact maintenance costs and issue and some extension blockers are among the main reasons that make companies to think about revamping the whole thing from bottom up.

I believe that it takes a whole new post analyzing this aspect of every re-design project. Anyhow, in the post above if you read between the lines I mentioned that maintenance costs and show stoppers play a major role in the renewal process.

First, let's look at it by comparing the proprietary and open source solutions from a maintenance point of view:

   1. License fees: I think the proprietary products stand no chance.
   2. Ease of customization and extension: Same as above. You can customize open source solutions much easier, even in house. Why? You have the source code!
   3. Cost of customization during maintenance period: Proprietary solutions are pathetic.
   4. Practical support: As far as I know for the money you pay to proprietary vendors you do not receive the support you deserve. In addition to that, regardless of the money you pay, the support itself is not as good as the support you can get from Open Source vendors and communities.
   5. TCO: No brainer, Open Source is the winner.

Verdict: Open source is more affordable and at the same time more extensible.

Well, I have not addressed your point yet. With this little background I think it would easier to explain what I think.

One of the reasons that cost of maintenance may go up is when you face hidden or new challenges with the new solution you have and you ask for extensions. This could not be eliminated due to the constant flow of emerging requirements; "What if we could do this that way?, Oh this sucks, it is really hard to do this simple thing this way, we should change it? ...".

You cannot help it, requirements keep coming. But you have control on how extensible you design the system to implement the new requirements.

Finally, I do not know a universal solid and practical way of quantifying advantages and disadvantages of keeping or revamping every component in your legacy system. Customers and Vendors are there to solve this problem. But one thing I know is that a generic, adaptable, pluggable and extensible solution, always stand a better chance for new changes.

 It became like a post itself!! :) Cheers.

Daniel Lourenço replied on Mon, 2010/05/31 - 1:25pm in response to: Ali Loghmani

Hi Ali

Thank you for your for the detailed answer. It was indeed a new post itself :). Although Open Source can be a space where you find full flexibility, I do not believe it is the only solution - the truth is that customizing and maintaining an Open Source solution can also be prohibitively expensive and such an option must be carefully analysed. But this would be an even bigger post :) . I work for OutSystems (www.outsystems.com) which has as one of the main focus TCO reduction. And in fact you can find in the market lots of proprietary solutions that aim at this same goal.

But we do agree that a "generic, adaptable, pluggable and extensible solution" is surely what you would want for a future proof application (especially if you care for the money you will spend keeping your applications up-to-date in the next 5 years). And I also agree that in some cases, Open Source solutions will have all you need for this.

Thanks again for the great answer!

Daniel

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.