Data migration is defined as the process of transporting data between computer systems, storage devices, or formats. While it is usually not part of a DBA’s daily routine, data migrations are one of those types of activities that most database teams are called upon to perform fairly regularly. In the majority of IT environments, migrating data is done for specific reasons that contribute to the strategic goals of an organization.
Types of Data Migrations
Data migrations are conducted for a variety of reasons. An enterprise may want to avail themselves of new technology, consolidate their resources, or upgrade to more modern hardware. Four general categories can be used to classify data migrations.
- Storage migration is performed when an organization refreshes its storage technology. This is usually done to take advantage of faster and more efficient storage devices.
- Database migration may be necessary when moving data resources to an alternate solution or to a new database that uses the same platform.
- Application migration can be done to move data within an application or when replacing currently used software with a new solution.
- Cloud migration is the process of moving data from an on-premises system to a cloud provider or between cloud vendors.
Required Steps for a Successful Migration
Approaching data migration haphazardly is a sure-fire recipe for disaster. A methodical strategy is required to address the complexities of migrating valuable data resources effectively. Here is an outline of a seven-step process that can increase the probability of a successful migration.
Pre-Migration Planning
Extensive planning before embarking on data migration is an essential first step in ensuring its success. There are many aspects of the project that need to be worked out before it begins. Here are some of the more critical items to address.
- Conduct a pre-migration impact assessment that considers the costs and probable outcome of the project.
- Notify all required stakeholders and technical teams of their roles in the migration.
- Identify the human resources that are required and when in the process they will be needed.
- Agree on the security standards or restrictions that affect the data migration.
- Develop configuration management and migration policies to eliminate unexpected roadblocks during the process.
Project Initiation
Once pre-migration planning is complete, project initiation is the next step. Part of this phase is to further refine the migration plans and ensure that the necessary components are in place. The following elements are among the most important areas demanding attention.
- Create a stakeholder communication plan and determine what type of reporting will be provided and its frequency of delivery.
- Define 3rd party supplier agreements if additional expertise will be needed for the project.
- Resolve all security issues revolving around legacy data so it can be accessed for the migration.
- Formally define hardware and software requirements for subsequent phases of the migration.
Landscape Analysis
As the migration planning proceeds, more details need to be ironed out and estimates can be solidified.
- Create a detailed data dictionary that catalogs all relevant information on the data being assessed for migration, which will facilitate team collaboration.
- Identify high-level source to target relationships that will be more fully analyzed in later phases of the project.
- Create conceptual, logical, and physical models that are necessary elements in defining the structure of the legacy and target environments.
Solution Design
In this phase of the project, the information gathered in the previous steps is used to develop the detailed implementation required for the migration. The following are some of the items considered at this stage of the process.
- Creating a detailed mapping of source and target objects is necessary so it can be used by developers using a data migration tool.
- Data quality management specifications are required to address issues with the data that were discovered in the landscape analysis phase.
- The hardware requirements for the new production environment need to be defined.
- Service level agreements with the businesses or business units that will be impacted by the migration need to be developed.
Build and Test
The building and testing phase of the migration is where everything needs to come together to position the team for success. It is an iterative step that involves the coordination of many moving parts and demands attention to a variety of details.
- Testing should be conducted using production data rather than a smaller sample set to reflect the conditions expected during the live migration.
- An independent validation engine needs to be devised that ensures the accuracy of the migrated data.
- A continuous data quality monitoring solution needs to be implemented that will carry over to the migrated systems.
- The development of a fallback policy in the event of a failed migration is an essential component of the testing phase.
Execute and Validate
The execution of the migration should demonstrate the culmination of the effort spent on the previous phases of the overall plan. The agreed-upon validation engine will inform the team about the success of the project. Logs should be kept to demonstrate compliance with any SLAs that are in place.
Decommission and Monitor
As the completion of the migration process draws near, there are important issues that need to be addressed. Retiring decommissioned systems should be done according to documented procedures and monitoring of the new production environment needs to be handed off to the operations team.
Choosing the Right Tools to Assist Migration
Having the right software tools will increase the chances of successful data migration. IDERA’s DBArtisan offers database teams a multi-platform solution to the problem of schemas and data. It supports Oracle, SQL Server, Db2, and Sybase from an intuitive GUI that employs powerful wizards to enhance productivity.
In particular, the Schema Migration Wizard can be an essential part of database migration. It uses advanced intelligence to resolve differences in database platforms, sequences object creation properly to eliminate dependency errors, and uses the fastest available method when copying table data. With a viable plan and the right tools, you will be well on your way to successful data migration.