The crawler could not communicate with the server
The crawler could not communicate with the server. Check that the server is available and that the firewall access is configured correctly
This is a rather generic error message and I found out that it generally covers problems communicating with the index server, i.e. the target server is responding with a http response code 5xx “internal server error” or not at all.
Quite often if I hit that particular page on the index server (not the WFE!) I would see the error. For instance in one site an email contact form was failing because it used the referral header that wasn’t given by the indexer, or if you hit it directly with a browser. Or an error in the web.config.
If you’re having this problem for local SharePoint sites (and you verified that the page worked) remember to test it on the index server, not just the front-end, as the index server is using itself for indexing. You might have forgotten to deploy some resources or lots of other errors are possible…… Enable stack trace on the index server (setting in the web.config) and fix the actual problem afterwards.