Sunday, February 1, 2009

This Application Has Unique Business Rule Needs

No it does not. If it did, then your customer/employer would be doing something no other human being has ever done, which is unlikely in the extreme. The application may be unique in its particulars, but it is almost certainly extremely common in its patterns. This week we will see how "unique" needs are in fact nothing but common ordinary development projects.

Beginning With the Conclusion

I have had this conversation with many programmers over the past few years, and it always follows the same patterns. The easy part of the argument is showing the programmer that what he thinks is special or unique is in fact common. The much harder part, because it involves the delicate human ego, is showing the programmer that he has not seen this because he is ignorant. This is not fun to do and I myself usually skip it, it's usually not worth the trouble.

Path 1: Details

Occasionally I speak to a programmer who thinks he has a unique situation. His claim begins with the mountain of details he must handle, details which appear to be contradictory, subtle, and overall perplexing. He wonders if some new approach is required to handle them.

In answering this claim, we begin with the easy part, showing that the situation is itself not unique. In short, all worthwhile projects involve mountains of detail, so there is nothing special there. When it comes to the subtleties and the maze of exceptions and special cases, these are common in mature businesses that have evolved this complexity in response to business needs over the years. So again there is nothing unique here, the programmer's situation is again common.

At this point we have to ask how the programmer will deal with this perplexing mountain of detail. If he knows what he is doing, he will give the general answer that he is going to break it down as much as possible into independent smaller problems that can be solved on their own. Since this is nothing more than how all programmers solve complex problems, the entire "uniqueness" claim has completely collapsed. His project is utterly common.

The much harder part of the conversation comes if the programmer does not know how to break down the problem. For instance, if the problem is all about a fiendishly complex pricing system with lots of discounts and pricing levels, and the programmer does not know that he needs to begin with the database, and he further does not want to hear that, well, there is not much I can do for him. He will end up working a lot harder than he needs to, and will probably remain convinced he is dealing with something "unique".

But let's go a little deeper into that example of the complicated pricing system. Why do I claim that he must start with the tables, and that is he is wasting time if he does not? Well, a complete answer is much more than will fit here, and in fact I hit that theme over and over in these essays, but it comes down to:

  • He must have an accurate and precise description of the details that govern the pricing scheme. That is what tables are for.
  • In working out the mechanics of the tables, particularly their primary and foreign keys, he will come to a his most complete understanding of the mechanisms involved.
  • When the tables completely reflect the details he must work with, the code will just about write itself.
  • Lastly, but probably most importantly, the customer will expect to control the pricing system by adjusting the parameters at all levels. Again, that is what tables are for. The user is in control of the pricing system if he can edit the tables (because of course he cannot edit the code).

Path 2: Combinations

Once upon a time we had simple desktop business applications, games, and then this weird new thing, "the web". Now they are all mixed together, as we play games on the internet that are tied into huge databases. Modern applications often combine technologies that used to be comfortably separate. On any particular project, some of the requirements look like they can be met with an RDBMS, some require management and delivery of media such as MP3 or video, and he is told as well he must provide RSS feeds and import data coming in XML format. Perhaps as well there will be stone tablets and papyrus scrolls.

This programmer may believe he is in a unique situation because of this combination of needs. Because no single toolset out there can meet the entire project, perhaps this is something never before seen? But this does not hold up. Just like the argument about complexity, he must break the problem up correctly, and when he has done so he will have a perfectly ordinary project. Though I might add it will also be a very interesting project and probably a lot of fun.

In The End It Is All About Patterns

I have given two examples above taken from my own experience where programmers have claimed to me that they faced some unique situation. There are many other cases, and they always make perfect sense to the person who thinks he has discovered something new. The biggest flaw in the programmer's thinking is failing to distinguish between particulars and patterns.

My claim in this essay is that the patterns of all problems are the same. Somebody has seen it before, somebody has done it before, the answer is out there. The process of analysis and programming is about slotting your particulars in the patterns that have already been established.

In the broadest sense all programs process data, and particular programs break down into broad patterns of data access and manipulation. Sometimes you have a broad range of users putting in data with very little massaging (think twitter) and sometimes you have one group controlling much of the data while others make use of it (think Amazon), and sometimes your data is mostly relational and table based (think any ecommerce or biz app) and sometimes its mostly media (think youtube).

Once you have these broad patterns identified, you can then proceed to make use of established practices within each particular area. What is the best way to provide sensitive data on the web and protect it from unauthorized eyes? Somebody has done it before. What is the best way to track large amounts of media? Somebody has done it before. What is the best way to set up a complex pricing system with lots of discounts and pricing levels? Somebody has done it before. In all cases, your particulars may be different, but the patterns will be the same.

Conclusion: Find the Patterns

Whenever I find myself looking at a situation that appears to be new, I try to tell myself that it may be new to me, but it is not likely to be new to the human race. If it does not appear to follow a well-known pattern then I proceed as if I have not yet recognized the pattern and continue to analyze and break it apart until the pattern emerges. So far it always has.

8 comments:

Ka Wai Cheung said...

Well stated...

I think you're right in that a programmer's first priority is to understand the data model behind the application (or on a micro-level, the problem). In fact, getting the model right automatically requires you to consider how these processes fit together (else you haven't considered the model enough).

I'd also say that on a microscopic level, there are often unique problems...which is why people that take known "design patterns" too rigidly get hurt. I do get the particular v. patterns point you're trying to make though.

Anonymous said...

stick to databases.

Starting by designing tables is not a good way to build an application. Start with a data model, sure, but that is not the same thing.

zippy1981 said...

Ken,

You need to place this article in places where programmers that believe their situations are unique congregate.

Regards,

Justin Dearing

Anonymous said...

Ignorance is bliss? I'd be interested in your thoughts on Zed Shaw's rant about the death of the ACL.

KenDowns said...

@gavin: my first comment is he probably could have said it all in 30 minutes :)

He had me right away with his remarks about MBA's who demand you solve problems and refuse to believe anything you say. But I thought it was rare and valuable that he gave so much advice about dealing with these people - about the human aspects and the politics of how software is done. Like it or not, these people are running the world, and you have to know how to deal with them.

But anyway, as far as ACL's go, I've so far only watched half of it (geez, 90 minutes altogether?) and so he has been bragging about his 400 lines of Ruby and his basic use of chained booleans, as soon as one answer is true the guy is in. In the context of my OP I would say he recognized the correct pattern, as borne out by the simplicity and speed of his solution, and its appeal to his users.

Also in the context of my OP, he never says the fundamental problems are new, he mentions that he is working in a field with plenty of competitors and well known requirements. What he does say is that he found a new approach to one of the fundamental operations in that field. That is cool, but it might be a stretch to try to fit it too far into my OP, since that is not what I was talking about.

Kelvin D. Meeks said...

Ken,

Perhaps I've missed something in what you've wrote, but would you generally agree that quite often the problem is misunderstood if the business process is not modeled accurately first?

From that you can iterate through the Domain Model, Use Cases - from which you can derive the entities and relationsihps...and eventually arrive at a Logical/Physical data model.

KenDowns said...

@Kelvin: The main thrust of the text is that once you understand the requirements -- by any route, situations which appear unique will turn out to be ordinary.

Going through business process to domains will get you there. I hold the unorthodox opinion that going from biz process to tables is better, skipping the abstraction of data, but that is a conversation for another day.

John Zabroski said...

Your forgetting to mention that often it is the MBAs in the room who think their business is unique, and the analyst's job is to politely convince them they're incorrect.