Speed of the CSV Import Process

This topic contains 5 replies, has 3 voices, and was last updated by  Stiofan O’Connor 10 years, 1 month ago.

We have moved to a support ticketing system and our forums are now closed.

Open Support Ticket
  • Author
    Posts
  • March 6, 2015 at 5:46 pm #33145

    headway
    Free User
    Post count: 49

    After having much trouble with CSV imports stalling and taking a long time, I got a dedicated performance server for two sites that are running GeoDirectory. I transferred over one site that already had around 375,000 listings added and another that had 0 listings added.

    Having a dedicated server with lots of RAM made it so that the imports are no longer stalling at all. It also made it so importing to the site that had 0 listings VERY fast. It can do almost 10 listing per second. However, the site that already has a large database is still taking a very long time to import. It is doing about 0.5 listings per second.

    Besides one having previously had listings added, they are both running on the same server, with identical set ups. Any ideas what can be causing this issue? Will the import process just slow down as the database gets larger? Is there any way for me to fix this?

    Thanks for all of the help so far.

    March 6, 2015 at 7:07 pm #33160

    Paolo
    Site Admin
    Post count: 31211

    Hi,

    I’m not sure why this is happening, so I’ve flagged this for Stiofan. Are the imported listings similar? Same amount of data, pictures and so on?

    Let us know,

    Thanks

    March 6, 2015 at 7:10 pm #33163

    headway
    Free User
    Post count: 49

    Yes, the imported listings are pretty much identical. There are no pictures.

    With the fast one, I did not manually create any categories before importing, as they are created automatically, which means I did not upload any place icon images for the categories, like I had done for some of the categories on the slow site. Perhaps that has something to do with it.

    Thanks.

    March 7, 2015 at 4:51 pm #33238

    headway
    Free User
    Post count: 49

    One other thing to note: for both sites, I can upload large CSV files, but if they are over 25k entries or so, when I try to import them, I get an error message saying that they are not valid csv files. They definitely are valid, and once I reduce the number of entries, they import just fine.

    With the shared hosting I was using previously, I was able to import 100k entries or so at a time (though it would pause often).

    Is there some setting I could/should change on my server/database to fix this?

    March 9, 2015 at 5:00 pm #33430

    headway
    Free User
    Post count: 49

    Update: the larger directory has pretty much stopped importing listings at around 430,000. It stalls every few minutes. It is on a dedicated server with lots of RAM and storage. Please advise on how I can go about getting more listings added.

    March 9, 2015 at 6:39 pm #33447

    Stiofan O’Connor
    Site Admin
    Post count: 22956
    This reply has been marked as private.
  • Author
    Posts
Viewing 6 posts - 1 through 6 (of 6 total)

We have moved to a support ticketing system and our forums are now closed.

Open Support Ticket
20% Discount Offer
Hurry! Get your 20% discount before it expires. Get 20% Discount