How to Import 9million records?

This topic contains 3 replies, has 3 voices, and was last updated by Stiofan O’Connor 7 years, 3 months ago.

We have moved to a support ticketing system and our forums are now closed.

Open Support Ticket

Tagged: import

Author

Posts
August 16, 2017 at 10:30 pm #391569

Utsav Sarkar
Expired Member

Post count: 29

Hi,

I am creating a directory site using geodirectory and I want to import more than 9 million records. The import function is super slow. Is there a way to quickly import all of these records?

August 16, 2017 at 10:35 pm #391570

Paolo
Site Admin

Post count: 31206

Hi,

the faster the server the faster the import, what are the specs of the server hosting this website?

Let us know,

Thanks

August 17, 2017 at 12:57 am #391588

Utsav Sarkar
Expired Member

Post count: 29

I am hosting on Amazon AWS m4.large instance.

The specs are:
2vCPUs
8GB Memory
Dedicated EBS Bandwidth: 450MBPs
2.3 GHz Intel Xeon® E5-2686 v4 (Broadwell) processors or 2.4 GHz Intel Xeon® E5-2676 v3 (Haswell) processors

Currently, it takes about 1 minute per 10 records. So 9 million records will take 625Days.. slightly less than 2 years! 🙂

Would it be possible to use phpMyAdmin to directly import the records into the database?

August 17, 2017 at 11:07 am #391650

Stiofan O’Connor
Site Admin

Post count: 22956

Hello,

9 million listings sounds great but to run a site like that you would need more that that hosting, if i were to run a site like that i would probably go with 1 mega dedicated server or maybe two good dedicated servers one for code and one for Database.

If you can give me some info about the project and budget i will give you my best advice.

Thanks,

Stiofan
Author

Posts

Viewing 4 posts - 1 through 4 (of 4 total)

We have moved to a support ticketing system and our forums are now closed.

Open Support Ticket

Hurry! Get your 20% discount before it expires. Get 20% Discount