Mr. Lott has been involved in over 70 software development projects in a career that spans 30 years. He has worked in the capacity of internet strategist, software architect, project leader, DBA, programmer. Since 1993 he has been focused on data warehousing and the associated e-business architectures that make the right data available to the right people to support their business decision-making. Steven is a DZone MVB and is not an employee of DZone and has posted 139 posts at DZone. You can read more from them at their website. View Full User Profile
is -- at best -- little more than an attractive nuisance. At worst, of
course, it turns into a maintenance nightmare and the reason for
killing the project. Let's look at the ways this can end badly.
Your "end users" can't or don't customize it with small scripts.
Bottom line: you wrote a scripting interface which only you can use.
Not all bad, but hardly worth the effort. Here's one version of
rule: "Make it possible for programmers to write in English and you will
find that programmers cannot write in English."
Your "end users" turn into the worst kind of hackers and break things
left, right and center. Bottom line: you spend more time debugging the
messes created by scripting.
3. The real world
use cases turn out to be more complex than you bargained for. You
spend endless time extending and modifying the scripting support to try
and balance limited power (to prevent problems) against enough
sophistication (to handle unforeseen use cases.)
you argue, "those are extreme cases. There has to be a middle road
where the user's behave properly." No, not really. This middle road
doesn't exist. Why? Programming is hard. A phrase like "simple
scripts" is contradictory.
War Story I --
The SAS Hacks
At one organization, the
accountants had created a monster. Their chart of accounts in the
General Ledger (GL) was not consistent across all of the organizations.
Consequently, financial analysis programs required careful
rearrangement of certain ledger totals to make them reflect the business
reality, not the legacy GL mistakes.
wrote some "simple" business rules in SAS do GL extracts. The extracts
which were processed by these rules were useful for financial reporting
because the legacy GL work-arounds were properly allocated.
job was to write a proper financial reporting tool that correctly
extracted and reported on ledger data. It was going to be a proper data
warehouse. The facts were amounts, the various dimensions were time,
account, organization, bank product line, etc.
because of the SAS hacks, we could not get a single straight answer to
any question. Everything had to be explained by a "here's some code"
answer. Their business rules numbered over 9,000 individual simple
Question: How is 9,000 of
anything "simple"? Answer: it isn't. The volume makes it complex.
-- of course -- the business rules were actually more complex than
claimed. The superficial claim was that the rules implemented
line-of-business (separate from organization) rules. Some statistics
showed that the rules likely contained two independent dimensions of
The "simple" rules were used to
create huge complexity. It's human nature. The world is complex; we
want to write software which reflects the world as we encounter it.
Story II -- The MS Access Hack
another organization, we had helped the end users buy really nice
reporting tools from Cognos (before
they became part of IBM) and Business
Objects (before they became part of SAP). The tools were -- at the
time -- the state of the art in flexible end-user reporting. Cool
drag-and-drop interfaces, flexible scripting add-on capabilities.
Did the users actually use this
dumb, large, detail-oriented reports. They extracted the bulk data into
MS-Access Databases. Then they hacked around in MS-Access to produce
the reports they wanted.
Why? (1) They
understood Access; learning to use Cognos or BO tools wasn't going to
happen. (2) They wanted real power; the limited scripting capabilities
weren't solving their problems.
The reason we
became involved was because their Chief Access Hack (CAH™) got another
job and no one could figure out how the Access stuff worked.
two weeks of reverse engineering, I found a bunch of business rules
that should have been in the data warehouse. We had a nice meeting with
their executive; we wanted to talk about the CAH and what they were
going to do to prevent this kind of MS-Access nightmare in the future.
executive reminded us that business rules are subtle and complex. Data
warehouse software development is slow and clumsy. The users will
always be several steps ahead of the warehouse processing. The
MS-Access hacking is how they coped with the limitations of the
is complex. We can't create a language suitable for "non-technical
users". Either the language is too technical for our users -- and it
isn't used -- or the users become technical and demand more features.
learn. They're only non-technical users on day 1. If they start using
the scripting language, they immediately start on the path to become
The real world is complex.
There are no "simple scripts" except in a few textbook cases. When we
first set out to build our simple scripting environment, we start with a
few sensible use cases. Then unforeseen use cases crop up and we
either tell people they can't use our system [Really, would you turn
them down?] or we have to add complexity.
Because of this complexity issue,
avoid some urges to refactor. Not all refactoring is good refactoring.
There is A
Limit to Reuse. Specifically, any attempt to create plug-in
scripting into a larger application rapidly becomes hard to manage.
It's better to provide necessary libraries to build applications around a
script than to plug a script into a standardized application.