Data Import Speed and Processor

This topic contains 1 reply, has 2 voices, and was last updated by  Alex Rollin 6 years, 11 months ago.

We have moved to a support ticketing system and our forums are now closed.

Open Support Ticket

Tagged: , , ,

  • Author
    Posts
  • November 22, 2017 at 2:41 pm #406398

    Brian Bateman
    Full Member
    Post count: 22

    I added GeoDirectory to my BuddyPress powered food blog at https://FoodSocialSite.com and have about 1/2 million restaurants, bakeries, etc. with geo coordinates that I can import.

    I started the first import of 44,000 bakeries and interrupted the import at about 1,400 listings because the site was really slow responding.

    The import was also taking a long time, so I didn’t want the site to be unresponsive for months, so I interrupted the import.

    My Question Is

    Does Anyone Know How to Do One of the Following?

    1. Make it “Drip” the place import at a rate of speed? ie: 1 per 30 seconds, 1 per minute, one per 3 minutes. If it has to be slow, let it be slow, but then slow it enough to not disrupt the front end.

    2. Import an InfoUSA CSV directly into MySQL? That should be fast and if it makes the site slow, at least it could be done in the wee hours and expected to be complete before the day’s traffic.

    November 22, 2017 at 2:56 pm #406402

    Alex Rollin
    Moderator
    Post count: 27815

    Hello!

    GD’s import/export system performs ajax validation calls: if the listing ID is already existing, how many images there are per listing and so on.

    It process 5 rows at a time. On our dedicated server it takes approximately 1.2 sec per listing. 140 listings where imported in a little over 3 minutes.

    1# There is no easy way to change the way the import process works. Most of the speed equation is determined by your hosting. Dedicated servers are recommended for the numbers you are talking about.

    2# That’s outside the scope of what we can help with here in support. But if you are seriously looking for help with importing all those listings, connect with one of the providers at http://geodirectoryexperts.com or post your project to the GD Jobs forum here.

    Related Links:
    https://wpgeodirectory.com/docs/server-requirements/

  • Author
    Posts
Viewing 2 posts - 1 through 2 (of 2 total)

We have moved to a support ticketing system and our forums are now closed.

Open Support Ticket
20% Discount Offer
Hurry! Get your 20% discount before it expires. Get 20% Discount