1.5 Million Record GeoDirectory!

Need help – we are creating a 1.5 MILLION record Geodirectory containing business information about real estate agents. It’s online at https://ethicalrealestatepro.com/

This is really slow on our server! We’re on a 16-core XEON 2.2 ghz server, with a full SSD hard drive and 64 gigs of RAM. It’s a BIG machine, and I’ve already allocated 1 gigabyte of RAM in PHP and the WP-Config file for our Geodirectory site.

The admin section of the site runs full speed, so the issue seems to be having to sort through all of the places when the front page is loaded. Any tips on speeding this up?

I’ve allocated more RAM to MySQL than a standard build, and I may bump that up further later today. However, are there any features that I can turn off to make this load faster? Any way to make it load partial data instead of the full-set when it’s pulling posts?

I’ve already done the easy stuff, including making the map run on the server side instead of the client side (which would be absolutely insane :o)

Ideas welcome! Help!