CSV Migration in Batches for Large sites

author details
AdiPie
8th Apr 2021
2 mins read
Image
CSV migration batches

Migration content from the current site to the external source can help to reduce the effort of the content author to prepare another site Launch. While these movements can fluctuate in type, normally accounting pages are sent out in a comma isolated worth (CSV) design because of their effortlessness.

While Drupal has strong help for moving in from a CSV file, the current structure can battle when given enormous CSV documents. Increasing the PHP memory limit for the migration was an underlying advance.

Therefore, we composed a custom Drush order that goes about as a covering around the default Migrate import command. Our custom command divides a huge CSV record into more modest documents that can be imported in clumps. 

drush migrate:import:batch migration_example --batch-size=50

At the point when the migration is run, the CSV source record for the sample_migration is part into more modest CSV documents with 100 lines each. The migration runs for every one of these files. These files are briefly put away in the private document's catalog and are tidied up after the migration is done.