Gunther Lenz on "Practical Software Factories with .NET"
In the recent article An Introduction to Software Factories, Gunther Lenz discussed the motivation for Software Factories, the basic building blocks, and how Software Factories can be realized. He showed the potential benefits this approach has for smaller software development projects.
I recently caught up with Gunther to delve a bit deeper into Software Factories and its relevance to software development teams today. In this interview, Gunther gives us deep insight into his experiences with Software Factories. Gunther is a Microsot ISV Architect Evangelist, and I used this opportunity to find out more about his role at Microsoft and the work of the Microsoft ISV Architect Envangelist team.
Aslam Khan: Firstly, I must thank you for making a chapter from your book "Practical Software Factories with .NET" available for download to the DZone community. Can you tell us a little bit about yourself, your past work at Siemens and your current role at Microsoft?
Gunther Lenz: Sure, I will start with my recent position and then work my way back in time. I joined Microsoft in the role of ISV Architect Evangelist for the NY/NJ/PA region in March this year. In this role I enable ISVs to efficiently architect, implement and deploy software solutions by fully, and efficiently, leveraging the potential of the Microsoft Development Platform. The goal is to accelerate the adoption of emerging Microsoft technologies to solve the business problems and/or open new revenue streams for the ISVs. As an Architect Evangelist I am not driven by revenue or product commissions which allow me to act as a trusted technical advisor, working to find the best overall solutions to the ISVs specific problems. Furthermore, I also act as a resource broker to assist in locating resources within Microsoft to meet needs of the ISVs. Typical examples of my engagement include hands on training, executive briefing sessions, architectural design reviews, compatibility testing, and proof of concept engagements.
Before joining Microsoft, I worked for more than 10 years for Siemens. I started to work for Siemens Medical Solutions, where I was involved in a large scale medical application framework first as software developer and then as project lead for development and test. In 2003 I started working as a senior consultant on software architecture and software factories at Siemens Corporate Research in Princeton, NJ. In this position I had the opportunity to learn about Software Factories and build a business case for our consulting offering to our internal Siemens customers. As a result of my consulting work, I published two books (".NET -- A Complete Development Cycle" and "Practical Software Factories in .NET") as well as numerous articles in magazines. Thus, I am also an invited speaker at international conferences and received the Microsoft Most Valuable Professional (MVP) - Solution Architect award in 2005, 2006 and 2007.
Khan: How did you get involved in the Software Factories space? Was it an "AHA!" moment or was it a progressive search for a solution to a problem?
Lenz: Good question ;-). Thinking back, it was a combination of both. I started looking into mechanisms to systematically reuse software as well as to automate the software development lifecycle at Siemens Medical Solutions around 2000. After some research and being involved in the Software Process Engineering Group (SEPG) I tried to apply these concepts to the (at the time brand new) .NET framework and captured those thoughts in the first book “.NET – A Complete Development Cycle”.
Once the book was published in 2003 I got to introduced to Jack Greenfield (the godfather of modern Software Factories and the co-author of the seminal book on Software Factories) who is working for Microsoft and we started a discussion on issues software development are facing today. We compared our approaches and it hit me with the “AHA” effect. It seemed just logical that Software Factories would be a more systematic way to accomplish what we were looking for. Furthermore it seemed that Software Factories are the next step in the evolution of software development to become a more mature (engineering) discipline. Based on these conversations I presented the ideas within the Software Engineering department at Siemens Corporate Research, which I joined in 2002. The idea was received as an interesting advancement for software development with my colleagues at Siemens. Therefore, my boss at the time (Juergen Kazmeier) granted me some freedom to research the area of Software Factories and to develop and implement a business case for a service offering from Corporate Research to other Siemens Business Units. Thus, the Software Factories research program was born at Siemens Corporate Research.
Khan: The word "factory" immediately conjures up images of massive industrial manufacturing plants. Many people think that Software Factories are only applicable to large organizations with many product lines. Can we use Software Factories for small teams with small product lines?
Lenz: Actually, that is an excellent point. The word “factory” gets associated with many things (in this case mostly negative) such as sweatshop, blue collar, large production facilities, monotonous work, etc. I experienced these perceptions many times. At some point I started proposing the Software Factories approach without mentioning the term Software Factories as such. This mostly eliminated the lengthy discussions and clarifications about the term itself. Instead, I used descriptive (mostly positive associations) words such as efficiency gains, systematic reuse, automation, modeling, and code generation. This resulted in a much more welcoming attitude, at least with most clients I worked with.
Personally, I believe that Software Factories are even more applicable to small teams than it is to larger development teams. This believe is based on my experience seeing more successful Software Factories, and Software Factories alike, implementations by smaller development teams, rather than by large development teams. One of the reasons for that might be the fact that smaller teams have to be even more efficient than larger teams, compared with the fact that in smaller teams the lead developers often take charge identifying and generalizing reusable parts with the responsibility of maintaining and extending these reusable assets clearly defined within the team. In fact, in a few cases, smaller teams did some Software Factories implementations without even knowing about it, or even more interesting, having some negative attitude towards the Software Factories paradigm.
In contrast, in larger organizations the approach often implies commitment from higher management levels to institutionalize the formal and informal structures and processes to support the development, maintenance, and extension of reusable assets by (in most cases) a centralized team.
Having said all that, obviously there needs to be a product line for the investment in reusable assets and automation to pay off over the long run. Obviously, the more investment in automation mechanisms and reusable assets is done the more reuse or efficiency gain must be realized to get to a positive ROI for the project. Another important constraint is the need for a well known, understood, and fairly stable domain, only then are we able to identify and implement assets that we can reuse and automate the development process.
Khan: You've clearly had considerable success with "running" Software Factories. In your previous article on DZone you show the break even point and when ROI is achieved. Did you ever doubt yourself and the applicability of Software Factories when you started with your implementations?
Lenz: It would be easy to say “No, I didn’t have any doubt” ;-), but it is not that simple. In the implementations I did, we were cherry picking. This means that I was lucky enough to pick the projects that had the best promise to succeed. The key for all project development technologies and techniques is to use sort of a checklist. This checklist helps to identify the relevant and best fitting tools, processes and techniques for a specific project in a specific environment. This point goes back to my usual disclaimer where I usually recommend the use of two tools that are available free for anybody to use: “Common sense and best practices”. Software Factories are not for every project, but using these tools with some sort of a checklist is invaluable to me in order to identifying potential candidate projects for the application of the Software Factories approach.
The main focus on the selection process was on areas where we can define a small domain that allowed us to generate efficient source code and artifacts with relatively small user interface. What that means is that we looked for domains with well defined variabilities that allowed us to generate implementations with very little user input. In the example published instead of implementing approximately 500 lines of code the developers only need to provide approximately 20 variables (and some of them are as easy to provide a unique name, pick from a pre-populated list-box, or drag and drop) to generate the same implementation.
Nevertheless, I was positively surprised by the results we achieved. Probably because of the experience that efficiency increase of a multitude, in our case greater than three, for software development are usually unrealistic. The fact that we could achieve these results especially with relatively low upfront investment in time and man power lead me to write the results up and publish them for reference and to make the case of Software Factories for other projects.
Khan: How do agile methodologies fit into the world of Software Factories? For example, do factories accommodate the avoidance of BDUF (big design up front)?
Lenz: Another very good and relevant question I get asked often. On the one hand, the very nice part about Software Factories is that they adapt to your software development style and process. This means that no matter if you use a very agile approach, or a more formal process, Software Factories will just fit nicely around it. On the other hand this also means that the use of this paradigm does not prevent you from doing a Waterfall process, including BDUF. It comes down to the software development team to define and use the best fitting approach for their need. Software Factories do not prescribe a certain process and will adapt nicely to the process used.
Interesting enough though, in our very first attempt to implement a Software Factory we almost fell into the trap of using a waterfall process. Eventually we realized that this does not work and that, in order to be successful, we need to adapt a more iterative approach that was more natural to the process used within the software development organization.
Therefore, we mostly used an iterative approach for our Software Factories development efforts. Another important consideration is the fact that most of the time we do not start out with a project on a green field. The positive side effect of this is that it enables us to extract existing assets and to refactor them, by generalizing and adapting them, to reusable components. This process usually starts with a fairly small feature set that is then incrementally extended. The key here is that you know the domain and you know which pieces are going to be used in future products, in defined variations. I just want to emphasize that you should not plan on implementing all common and possible variable features upfront, rather focus on a subset and grow the collection over time. This means you start with a small set of common and variable features as reusable assets. In the product development extensions and additions are implemented. Some of these extensions and additions will be pulled into the pool of reusable assets in later iterations. Thus, the Software Factories common and variable assets will grow over the course of many iterations.
It is also important to have a feedback loop from the users of the Software Factories to the Software Factories developers. In smaller teams this is easy since it is most likely that the users of the factories are actually the Factories developers, eliminating the communication gap that exists in larger development efforts.
Khan: What is the main barrier for someone trying to implement a software factory in their organization?
Lenz: You mean besides the term Software Factory ;-)?
Seriously, there are many different barriers to overcome. For example, in larger development organizations adoption of Software Factories requires buy in from upper management and organizational changes. Why is that? Because we need someone responsible for the reusable assets, someone needs to decide which features become Software Factories assets, someone has to maintain the assets, and someone has to extend the existing asset base. In larger organizations this is usually done by a centralized team and with that centralized team comes a lot of responsibilities. Furthermore the development efforts have to be synchronized, and communication channels have to be established. This goes hand in hand with fighting the not invented here syndrome, where one team rather builds their own implementation than reusing modules developed by another team. At the end all this comes down to an additional investment with the goal to get more efficient and to increase quality and that is a tough sell since most organizations rather innovate on the promise of cost cutting then efficiency and quality gain (see also my blog post http://blogs.msdn.com/glenz/archive/2008/06/02/what-drives-the-adoption-of-new-technologies.aspx ). This turns actually out as a huge barrier and it was one of the main reasons we tried to show the quantitative business value with our publication on MSDN (http://msdn2.microsoft.com/en-us/library/cc496679.aspx).
In smaller teams I found that there are actually less barriers because no organizational changes are made and developers naturally try to improve efficiency and reuse, so giving them a structured way to do this in the form of Software Factories is usually embraced. I successfully worked on Software Factories with small teams where Software Factories definition, implementation and use stays within one team and no organizational change or upper management involvement was necessary and upfront investment was minimal.
Khan: With everyone climbing onto the SOA train, how can we use Software Factories for churning out Services, if at all?
Lenz: We absolutely can leverage Software Factories for SOA development. In fact, the patterns and practices group at Microsoft published the Web Services Software Factory which you can download for free from codeplex at http://www.codeplex.com/servicefactory. The Web Service Software Factory (also known as the Service Factory) is an integrated collection of tools, patterns, source code and prescriptive guidance. It is designed to help you quickly and consistently construct WCF and ASMX Web services that adhere to well known architecture and design patterns.
While this is a horizontal Software Factory that enables the rapid development of web services, there are also opportunities to provide more specialized Software Factories for certain domains. These factories could be build from scratch or simply take the Software Factories from the pattern and practices group to specialize it. There are complementary Software Factories available from the patterns and practices group for WebClient, SmartClient, and MobileClient development and there is also some thought on providing a Software Factory for the development of Silverlight based Web 2.0 applications.
There are great opportunities for ISVs and third parties to provide a tremendous value with Software Factories for different domains in the SOA space. I see lots of significance for Software Factories in the Web 2.0 arena to support the development of applications that are orchestrated from many services, provide a rich user experience, and pull data from many different resources. Enabling an easy and very efficient way for bringing such applications to market has lots of potential.
Khan: Another other current topic is SaaS (Software as a Service) offered by ISV's. Do you see Software Factories gaining momentum because of things like SaaS and SOA?
Lenz: At Microsoft there is a lot of talk around S+S, which is an extension of the SaaS strategy by promoting solutions of an ecosystem that lives in the cloud and on the desktop. For this type of ecosystem best practices are published in form of blueprints at codeplex. These blueprints are codified best practices that support the workflow very similar to what I would call S+S Factories. As I mentioned before, I think the ability to orchestrate applications and connecting different services to create business value lends itself for a Software Factory approach. I see more traction with projects that are targeting the S+S space, but were not involved in any implementation, yet. I would love to hear if there are projects out there using the Software Factories approach for S+S development.
Khan: You are now a Microsoft Architecture Evangelist for ISV's. In your opinion, what are the challenges facing ISV's today?
Lenz: This is a very difficult question, since the problems often are related to specific business areas and the situation of the ISVs we are talking to.
Nevertheless, I think, in general, there are two different sorts of challenges. One is a technological challenge, and the other is a business challenge that the ISVs are facing. From my conversations with ISVs it seems on the technical side it is very difficult to keep up with new technologies (actually, that is already difficult working for Microsoft and keeping up with what product is released when and with what features), the roadmaps, and the migration path to leverage new technologies and techniques.
- Keeping up with roadmaps is difficult with vendors such as Microsoft, but it gets often more difficult if open source components are involved. Based on the feedback we hear from the community we try to address points we hear often such as providing access to expert resources in visual design with WPF for those ISVs that are currently looking to re-platform and providing easier access to partner resources, and understanding how to get licensing questions answered, leverage partner benefits, how to get into the partner program, what benefits exist, etc.
- Furthermore, very often we hear the ISVs asking us on advice on how to move from their current solution to a new technology or paradigm such as SOA, SaaS, or S+S. In all these cases we try to help the ISVs with insight into the product roadmaps, Architectural Design Sessions, and Proof of Concepts in Microsoft Technology Centers. In fact we just started to formalize the process to synchronize the Microsoft product roadmap with the ISVs technology needs. We hope that this will help to remove some of the challenges for the ISVs.
- Another challenge on the technical side is to attract and retain talented people. There is a lot of demand for highly skilled developers, architects, testers, and project managers and this leads to tough competition between the ISVs for those scarce resources. We try to enable the ISVs to educate their employees with the latest and greatest technologies by providing briefing sessions at the Technology Centers, providing training in new technologies, as well as the participation in early technology adoption programs.
- The last major technological challenge that I wanted to mention in this context is the larger companies moving into the core business of small, or medium, sized ISVs either by introducing competing products and/or by acquisition of smaller players and integrating the acquired products into their product suite. We address this by building strategic partnerships with the ISVs, so they can leverage Microsoft initiatives for business development in addition to the technical content.
On the business side it is most important for the ISVs to distinguish themselves from the competition.
- Very often this is done by providing innovative, richer, and more efficient user interfaces. In order to achieve that ISVs align with big vendors and take advantage of their sales challenge.
- Many of the ISVs I work with are very successful by making use of the Microsoft provided sales and marketing partnerships.
- For small ISVs and start-ups there is certainly the challenge of acquiring funding part of the main difficulties. We try to provide technology to these companies through the empower program and initiated the incubation weeks, where entrepreneurs had the ability to get technology support and present their business ideas to Venture Capitalists and Angel investors.
Khan: What can we look forward to from yourself and the rest of the team?
Lenz: We are trying to be more open and to engage more closely with the ISV developers. There is many initiatives that we are currently undertaking. For example, we have Facebook site with a neat application where developers can connect depending on their technology interests.
We also started to do webcasts, targeted to the ISV community. The first of such events was on Composite Rich Internet applications. If you are interested check it out here.
One of our newest initiatives is to help start-up companies and microISVs. The goal here is to provide relevant technical resources, such as Software and other resources, as enabler for the start ups (http://www.empowerforisvs.com). This initiative builds on the success our team had with the incubation weeks that we had in the last few weeks. If you are interested, then check out this video. We are looking to extend these efforts for start-ups with a podcast series that will provide information on the business relevance of technology decisions for the success of start ups. I am currently trying to define the strategy for that, so if you have ideas or suggestions, let me know :-)
In addition to that we have our blogs with news from all of the team members and personal blogs (you can find mine at http://blogs.msdn.com/glenz ). We are trying to reach out and are always looking for the best way to reach out and engage with the ISV community. That was one of the reasons why I started to contribute to DZone ;-). Getting involved does not only provide ISV developers with information from Microsoft, but it also enables us to get feedback from the community that we can use to improve. Again, if you have ideas or suggestions on how we can improve these efforts, let me know.
Khan: Gunther, once again, thank you for your time, for your kind permission to provide a chapter from your book and your contributions to DZone. Good luck with your work at Microsoft and we look forward to hearing more from you and the rest of the Microsoft ISV Architect Evangelist team.
Lenz: It's a pleasure and thanks for the great questions, Aslam.
(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)