Unless you are starting a project or business from scratch, moving to the cloud will require some form of data migration from your existing infrastructure to your new cloud platform.
Microsoft Azure has a variety of tools to assist with copying your data from your current platform to their public cloud service. These range from powerful command line tools to copy unstructured data to dedicated tools for database migrations. However, before we look at the different tools that help migrate your data, we need to consider the actual data types you will be migrating.
Understanding data types is essential to migration success
Before we look at the different types of data migration options available to you, it is important to understand the different types of storage available on Azure and the ideal function of each storage type.
There are multiple data types supported by Azure. These range from structured data such as relational database management systems which support data stored in SQL databases to unstructured data which can be stored in blobs, files, queues or NoSQL tables
The type of migration you will be undertaking depends on what kind of storage you will be copying your data to, so you should understand these basic concepts before you start the migration process. Selecting the right type of data store is an essential precursor to a successful migration. The data store you choose forms the foundation of your cloud solution and the migration to each type differs from store to store.
Five useful tools to migrate your data to Azure
Now that we have an understanding of the different data types and how Azure stores this data, let’s look at the tools available for data migration. As there are different data types, ensuring you select the right tool is essential to your migration success.
Tools for migrating unstructured data
Unstructured data migrated to Azure will ultimately need to be stored in an Azure storage account. Azure storage accounts have three different types of data services each with its own unique set of characteristics which you’ll to need to align with your data use cases. Azure blob storage is optimized for storing massive amounts of unstructured data, such as text or binary data while Azure file storage lets you set up a file share in the cloud. Finally, Azure queue service stores messages for event queues such as those created and consumed by enterprise service busses.
There are multiple tools which help you manage data in an Azure storage account. These include Azure Storage Explorer which has an easy to use graphical user interface to command line tools such as AZCopy, Azure Powershell, and the Azure CLI.
1 – Azure Storage Explorer
Azure Storage Explorer allows you to easily manage your Azure storage accounts via an installed client-side application which runs on Windows, MacOS, and Linux.
Migrating data with Azure Storage Explorer is as simple as copy and pasting files into a standard operating system file system manager. Naturally, the larger the size of the data and the speed of the network will impact how long it will take to copy data from the source location to your Azure storage account subscription.
Command Line Tools for migrating unstructured data
2 – AzCopy
AzCopy is a command line utility built specifically to aid users in copying data from external sources to Azure storage. Also, AzCopy also lets you copy data from one Azure storage account to another and even copy data across subscriptions.
This command line utility, which in essence is the stripped-down copy functionality from Azure Storage Explorer, is available for both Microsoft and Linux platforms. To use this utility, you will need some familiarity with command line syntax which makes it a little less simple to use than Azure Storage Explorer. However, as it is a command line interface, it can be installed on systems which do not have a graphical user interface such as Windows Core, Windows Nano, and Linux servers.
3 – Azure PowerShell
PowerShell is a command line scripting language built by Microsoft which gives you the ability to interact with Microsoft technologies via a task-based interactive shell. Azure PowerShell is a set of PowerShell cmdlets created to communicate with Azure Resource Manager letting you manage Azure services via the command line.
Azure PowerShell, like Azure Storage Explorer and AzCopy, allows you to manage the transfer of data to, from, and within Azure storage accounts. With Azure PowerShell, you can script an entire data migration by creating a storage account for copying and verifying the data.
4 – Azure CLI
Like Azure PowerShell, the Azure Command Line Interface (CLI) provides a command line and scripting environment for managing Azure resources. However, unlike Azure PowerShell, the Azure CLI also runs on MacOS and Linux giving you the flexibility to interact with Azure via the platform of your choice.
As with AzCopy and Azure PowerShell, using the Azure CLI requires knowledge of the commands and syntax needed to execute tasks on the Azure services. The Azure CLI can be used to migrate data and, as with Azure PowerShell, the entire migration process can be scripted.
Migrating Structured Data – Tools for Migrating SQL Databases to Azure
5 – Microsoft Data Migration Assistant
Migrating a SQL Database from a SQL Server to Azure SQL involves several steps made easier with the aid of the Microsoft Data Migration Assistant. This tool lets you check the source and destination databases for compatibility issues and then manages the data migration for you after executing the necessary prerequisite scripts and backups.
Unlike the migration of unstructured data, which in essence is copying data from one storage location to another, migrating a SQL Database involves a number of predetermined steps to ensure the Azure site is ready to accept the structured data. However, the same data volume and network limitations exist. The larger the volume of data and the slower the network speed, the longer it will take to copy the actual data and complete the migration.
What to do if your data volumes are too large
Any data migration, be it structured or unstructured data, involves copying data from its source to Azure. The larger the volume of data, the longer the migration process. Furthermore, the speed and capacity of the network carrying the data determine how long the data copy process will take to complete.
Waiting days or weeks for data to copy is often not an option. To aid customers migrating their data to Azure, Microsoft has created Azure Import/Export so you can ship physical disks to Microsoft who will upload the data for you saving time and bandwidth.
Remember – Plan before you execute
Migrating data to Azure can seem like a monumental undertaking, but if planned and executed correctly the process can be done with little effort.
To make sure your data migration is a success it is essential first to plan the actual migration taking careful note of the type of storage you will be using on Azure. Once this is determined you can choose to use a graphical interface tool such as Azure Storage Explorer or one of the many command line tools available. Finally, if your network does not have the speed or capacity to handle the volume of data you intend to migrate, you can make use of Azure Import/Export to securely copy your data to its final destination.