Wednesday, January 16, 2013

All Software Development is Schema Management

Before you automatically disagree, stop a bit and think about it.  Can you think of any code you have ever written that did not handle data of some sort?  Of course not.  This is not about relational data either, it is about any data.  There is no software that does not process data.  Even the textbook example of some function that squares a number is processing the parameter and returning a value.  The first line of that function, in most languages, names the input parameter and its type.  That is a schema.

This remains true as you climb out of the toy textbook examples into simple one-off programs into small packages and ultimately to apps that build to megabytes of executables running on a rack (or a floor) of servers.  Just as each method call has parameters, so does each class have its attributes, every table its columns, every XML file its XSD (or so we hope) and that code works if and only if everybody on the team understood what was supposed to happen to the data.

Are We Talking UI or Platform or Middleware?

Generally today we are talking about the server side of things.  This is about any project that is going to take a user request and consult a data store: the file system, a search engine, a database, a NoSQL database, or an XML database.  If you go to the disk, that is what we are talking about.

New Versions Are (Almost Always) About Schema Changes

So imagine you've got some working code.  Maybe a single script file, or perhaps a library or package, or maybe some huge application with hundreds of files.  It is time to change it.  In my experience new code means some kind of change to the schema.

I cannot prove that, nor do I intend to try.  It is a mostly-true not an always-true.

Schema Here Does Not Mean Relational

This is not about relational schemas, though they are included.  If you are using Mongo or some other NoSQL Database which does not manage the schema for you, it just means you are managing the schema yourself somewhere in code.   But since you cannot write code that has no knowledge of the structure of the data it handles, that schema will be there somewhere, and if the code changes the schema generally changes.

Does It Need To Be Said?

Sometimes you will find yourself in a situation where people do not know this.  You will notice something is wrong, they first symptom is that the conversation does not appear to be proceeding to a conclusion.  Even worse, people do not seem to know what conclusion they are even seeking.  They do not know that they are trying to work out the schema, so they wander about the requirements trying to make sense of them.

Order and progress can be restored when somebody ties the efforts down to the discovery and detailing of the schema.  The question is usually, "What data points are we handling and what are we doing to them?"


free softwares said...

Thank you for making the effort and spreading this information with all of us. It was indeed very useful and informative while being straight forward and to the point.
Freeware Software

Noman Khan said...

Raj IT believes in building long lasting strategic alliance with its clients. We provide easy solution to make things lot easier. Our slogan (Making IT Easy) says it all. We do not predict ourselves as a provider of a solution for our clients; we are companion to the process that seeks the most effective solution paving the journey to a better tomorrow.
Web Design
Software Developments
Web Development
SEO Expert
IT outsourcing
Software Design & Development Bangladesh

Derek Craig said...

This continues to be real as you go up out of the toy publication illustrations into simple one-off applications into small offers and eventually to applications that develop to mb of exe operating on a holder of web servers.

Chris Saxon said...

I think your final question in this post is a great point Ken. Not enough people ask this however!

Instead I think many focus on what the application should do, which leads to the same data items being defined in subtly different ways. Which of course causes difficulties when trying to consolidate various pieces of code behind services, as I'm doing at the moment.

Clara Snyder said...

I’ve read several good stuff here. Definitely price bookmarking for revisiting. I wonder how a lot effort you put to create this sort of wonderful informative web site.
Oracle SQL

Shim said...

This is really a helpful blog i am really impressed with your work bro , keep it up the good work (^_^)

Ultimate programming tutorials

sanjay varma said...

Appreciation for nice Updates, I found something new and folks can get useful info about BEST ONLINE TRAINING

Brave Boss said...

Great writing! I have a lot of admiration for your writing. Thanks for all your valuable input on this topic.
Database Development Company NJ

John said...

This is an interesting way to look at the way we perceive software development, indeed it is all about data and you are correct when you emphasize the importance of schema, great read!

vijitha said...

Hey, nice site you have here! Keep up the excellent work!

Iphone Application Development

OfficeSpace said...

thank you for sharing

singapore jobs dubai jobs abu dhabi jobs gulf jobs asia jobs south east asia jobs apac jobs india jobs sri lanka jobs asia job board singapore job board india job board china job board sri lanka job board japan jobs japan job board singapore careers india careers dubai careers gulf careers asia careers apac careers china careers south east asia careers east asia jobs china jobs asian jobs singaporian jobs chinese jobs indian jobs middle east jobs middle east careers asia recruitment singapore recruitment sri lanka recruitment india recruitment china recruitment apac recruitment east asia recruitment south east asia recruitment expat jobs recrutement au maroc emploi au maroc recrutement algerie emploi en algerie recrutement en tunisie emploi en tunisie africa jobs africa careers europe jobs europe careers usa jobs european jobs australia jobs canada jobs dubai job board middle east job board gulf job board free job posting sites

Sara Taylor said...

I’ve read several good stuff here. Definitely price bookmarking for revisiting.
web design dubai

Erfan Abdi said...

Very very usefull
thank you very much

Tia Sopyan Hidayat said...

Programer yeah !!

IT Services said...

Appreciation for nice Updates.Thanks for all your valuable input on this topic Database.

Oracle Certification said...

Great work Sir related to "All Software Development is Schema Management"

Qadir Shaikh.
Visit at

oracle apps training said...

Such a nice blog you share....
Thank you to know about this....
Qadir Shaikh.
Visit at

Bruce Wayne said...
This comment has been removed by the author.
Bruce Wayne said...

thanks for help for software development and very useful for database programmer if you are beginner for any Online tutoring, programming help, assignment help, homework help, online programming, online assignment help all kind of help are provided by many site they will provide you Programming help and Assignments help is one of the famous service provided by them. Moreover they provides report writing help at cheap prices

Anonymous said...

Create your own site, avoid blogs. What all you need is router firewall, switch, SERVER. Run it 10 hours a day from home.
Register to

Network Security, UTM, Next Generation Firewall, Endpoint Security Appliances. FortiGuard Center; Fortinet Blog; Video Library; Contact Us; Products. Network Security.

mail me :

all white said...

Fact!!! Thanks for sharing complete point…one more company has articles about Webdesign Agenter website pixo web design.

Kim said...

Your argument is excellent!! Unique points in the same, here one more website I like joomla development company.

Joshua Thai said...

I really appreciated with your content and completely agree with you, here also I was visit a company website Digital zona.

Robert Key said...
This comment has been removed by the author.
Robert Key said...

Thank's for sharing the useful information!Cat5e plenum

Softwarehatch said...

FileMaker PHP London, database creation and information architecture.

Sophia Right said...

This is my first time i visit here. I found so many entertaining stuff in your blog, especially its discussion, I guess I am not the only one having all the enjoyment here! Thanks

Sophia Right said...
This comment has been removed by the author.
Data visualisation said...

Data Warehousing and mixed database workloads. It’s a preconfigured package of software,
servers, and storage that features Oracle Real Application Clusters, Oracle Automatic Storage
management, and intelligent Oracle Exadata Storage Software for consolidating databases on
the cloud with the simplicity of an engineered system.

Elisabeth said...

Could you please delete the comment above with the timestamp August 9, 2013 at 5:18 PM

It's a spam