Starting in September of 2006 and for about one year I took a hiatus from my career as a Data Warehouse focused Data Architect, Data Modeler and ETL Architect.
As I prepared to re-enter the market as an BI consultant I started reading articles about all things IT especially database and BI related. I was reading an article about the lack of version control systems in many IT shops. The author was saying that to his amazement so many companies still go without a version control system of any kind, and that seemed crazy given that Subversion aka SVN and other great tools are open source and hence “free”.
Sure enough when I got to my next client, a $4 Billion a year manufacturing company, the global BI group was not using any source code control for their database code or their data models. My first role with this team was as Data Architect/Data Modeler responsible for generating reporting data models and the DDL required to create the tables, views, partitions, indexes ect. for the Datastage ETL developers to load and the Business Objects developers to build reports from.
Finding no Version Control System (VCS) and a very manual process for configuration management and recalling the article mentioned above I downloaded my first copy of TortoiseSVN (Windows integrated SVN client) and started reading the online manual. Initially I used a file based repository on my laptop”s hard drive, then as I learned a little more I put the file base repo on a network shared drive that was being backed up.
The other issue we had was change management (new requirements, issues, defects) was being handled in email. When the database needed to change due to a request from an analyst or ETL developer or from a refactoring after a design review, there was very little documentation of who, what and why. I was first asked to generate a “paper” change request system, a form to be filled out whenever someone wanted a change. Trying to get all the right things on the form and then route and process the form via email was just making things worse.
I recalled that while at another very large financial client, we had managed a truly amazing amount of change with another tool/application called Mantis. Mantis is an open source tool (free) that was designed as a bug tracker. It allowed us to create issues, defects, requests etc. that were managed in a central database. Each item could be assigned to specific team members and then re-assigned to flow the work item through a process, allowing us to track who requested what and when, and then who changed it, approved it, deployed it, loaded it, and developed the report for it. Next using the Mantis Item number in the Subversion commit message allowed us to link code changes back to request and defects and we had the basis of a poor mans Application Lifecycle Management system.
We conducted a proof of concept with Mantis and Subversion running on someone”s desktop. Eventually we made the case to management that it should be installed on a server that would get backed up and supported. At first we requested to share resources with existing servers but none of the admins knew enough about SVN or Mantis that they wanted it on their servers. No thanks, too risky, not on my box.
Part of the problem of trying to find a host for our SVN and Mantis was that each required several parts, a LAMP stack (Linux, Apache Web Server, MYSQL database and PHP), which prior to virtual appliances had to be installed, configured and maintained separately for each application. Enter Virtual Appliances and Jumpbox.com.
Then we discovered that we had a nascent but growing VIrtual Machine infrastructure and “virtual appliances“. According to Wikipedia:
“A virtual appliance is a pre-configured virtual machine image, ready to run on a hypervisor; virtual appliances are a subset of the broader class of software appliances. Installation of a software appliance on a virtual machine and packaging that into an image creates a virtual appliance. Like software appliances, virtual appliances are intended to eliminate the installation, configuration and maintenance costs associated with running complex stacks of software.”
Jumpbox.com offered virtual appliances for both Mantis, and SVN, each in its own appliance, preconfigured, plug and play for less than a the price of taking the team out for a sushi lunch.
Additionally Jumpbox pricing was for an unlimited number of instances, so we were able to create dev, test, and prod instances of our appliances so we could follow our process with the tools as well as for the applications we were supporting.