Amazon Relational Database Services (Amazon RDS) is a tool that simplifies the setup, operation, and scaling for your web application’s data. Once it’s set up, there’s a lot of benefits including performance, security, and backups.
However, admittedly, data migrations and AWS RDS is probably the most complicated part of an end-to-end migration as it includes a lot of critical and secure data! Today, we’ve summed up some of those common gotchas that we’ve come across while migrating data to Amazon RDS.
1. Amazon DMS is NOT always the best tool
Amazon’s native data migration DMS is the best tool out there… if there are no data type or version compatibility issues. In other words, if you have a basic database, the tool does exactly what it's prescribed to do. However, throughout our experiences, we’ve found that a lot of older databases are just not compatible with Amazon DMS.
For example, when it comes to databases that use MySQL or PostgreSQL, stack.io uses alternative workarounds to make the data migration more smooth!
MySQL. Our manual process for replicating data in a MySQL database is a bit more tedious, but less prone to error and much more reliable than Amazon DMS. The gist of the replication is as follows. First, you dump the source database. Then, you move the dump to a target database and perform a restore there. Finally, you set up a sync between the source and the target using
mysql.rds_set_external_masterPostgreSQL. stack.io uses a tool called pglogical. It is a logical replication tool made by former PostgreSQL developers made specifically for PostgreSQL. pglogical uses a push/subscribe model to perform selective replication.
2. Prepare your on-prem for migration
RDS will likely not be compatible with everything you have on your on-prem database. In fact, some of the older extensions or versions of your database components might outright not be supported. In order to address this, you can take a data dump, load it into RDS, and verify that everything is working / resolve any issues before performing a live replication.
Then, when you’re ready to migrate your production environment, make sure you set up a live replication to avoid application downtime. Make sure that you choose the appropriate replication instance. Some factors to consider are tables size, data manipulations language, and total size of the migration.
3. Overestimate the time needed for migrations
Database migrations will always take time, even for smaller databases. The wide range of configurations that a database may have can sometimes add unforeseen delays in migrations. As such, you should always, always overestimate the time needed for migrations.
In an early attempt to use Amazon DMS to migrate from PostgreSQL, a migration took multiple days to perform due to an inaccuracy in Amazon’s documentation. (This in itself, is another gotcha! For complex documentation, always refer to multiple sources for confirmation. Documents sometimes are simply outdated or inconsistent. Remember, if you find an error in documentation, make sure to report it or submit a change request!)
4. Make sure your database is not exposed to the public internet
Although AWS RDS comes with some default security features as well as demonstration of compliance with applicable legal and regulatory requirements, there are always extra steps that you can take to ensure the security of your data.
One common mistake is exposing your database to the public internet. Under no circumstances should it be necessary to expose your database to the public internet. If you need to communicate with the database from a VPC, you can set up VPC peering to allow communication. If you need to talk to the database from outside of RDS, there are multiple options available such as using a bastion host as a proxy for your database.
Conclusion
Although the initial data migration to Amazon RDS is complex, a cloud-based database will bring a lot of benefits to your web application. Migrate your data to the cloud to unlock better performance, enhanced security, automatic backups, and more.
What is the state of your database infrastructure? Let us know.