As the Technology is growing many Systems or Data bases are
getting Converted / Migrated from Legacy Systems to New Systems in the
Market. So here we should be aware of
how data is getting Migrated /Converted and how this can be achieved with
Manual and also with the help of Tools.
This blog major describes about the Data conversion and
Migration and what is the Testing Scope.
To clear what is the difference
between Data Conversion and Migration……
Data
Conversion Testing:
Data
conversion is the transformation of data from one format to another. It implies
extracting data from the source, transforming it and loading the data to the
target system based on a set of requirements.
Data
Migration Testing:
Data migration is the process of transferring data between,
Old to New Systems for E.g.: Oracle 10
to Oracle 11g, Informatica 8.6 to Informatica 9x version Therefore data
conversion is only the first step in this complicated process. Except for data
conversion, data migration includes data profiling, data cleansing, data
validation, and the ongoing data quality assurance process in the target system.
Data
Migration Testing is involved when you want to move the data from Old Database
i.e. Legacy system to New Data i.e. Target System and also called the
Destination Databases. The Migration Can be done by ETL Tools (Extract
Transform and Load) either Informatica, DataStage, Abinitio etc. The ETL Tools
may incorporate some business rules to increase the quality of the data moved
to the Target System.
Flow
Diagram
Types of Data
Migration:
Database Migration:
When a data is migrated from existing database vendor to other database vendor or existing data base is updated to latest version. E.g.: Oracle to Tera-data.
Data center
relocation:
When a data center is moved from one location to other location, we need to migrate data from legacy data center database to target data center database.
Application migration:
When an application is migrate, for example migration from an on-premises enterprise server to a cloud environment or from one cloud environment to another, underlying data also need to be migrated to new application.
Risks Involved in Data Migration Process and solution to overcome.
Data loss risk
When data available in
legacy system but after migration process, if it is not available in target
system, then it is termed as data loss. Data loss is highest risk in data
migration. Cost involved rectifying data loss and business cost involved due to poor
data adds up as to financial and reputation risk.
Solution for the Above
Risk:
Here Reconciliation
will come in to Picture for the above Risk. In the process of extracting data from one source
and then transforming the data and loading it in to next layer, the whole
nature of the data might change. It might also happen some important data might
may loss. The Reconciliation process helps you to identify the issues. In Reconciliation we are going to validate
high level scenarios.
- Total count of rows, example
- Total
Customer in source and target
- Total
number of Products in source and target etc.
- Total count of records based on a
condition, example
- Total
number of active customers
- Total
number of inactive customers etc.
- Total
number of Customer based on the location/region etc.
The following are the bullet points that include Scope of
Data Migration:
- All the Configuration Tables should be up and should be loaded before the Primary Modules get Loaded.
- All the Users should get converted from Legacy Systems to New Systems.
- Should understand the functionalities between the Legacy Systems to New Systems.
- Data should be cleansed until 100% data get converted.
- Every Attribute of Present in Source Database should be loaded into the Target Database.
- All data related to a particular entity is loaded in each relevant table in the target database.
- Each business rule should be correctly implemented into the Target database with the help of ETL Tool.
- The Data Migration should process with expedite process without any major bottlenecks.
- Data should be in Sync before validating the converted are not migrated.
- Should increase the performance while loading the data to the migration systems.
- SQL Queries should be tuned at every level.
- Should validate 100% Data Comparison.
- Should Validate the Data Types.
- Integration Testing should be done after getting migrated weather all the systems are working properly as per the requirement.
Risks:
·
During
the Migration sometimes the meaning of legacy columns and Target columns holds
the same meaning but values are not going to match for e.g.: Monetary fields
are not going to match.
·
During
the Migration many data quality issues will encounter should be cleansed
properly before the data getting migrated.
