The SharePoint Swiss Army Knife

Environment:

SharePoint 2010 Enterprise
3 WFE’s each with Query Roles
2 App Servers with 1 Dedicated Indexer

Error in Event Viewer:

Event ID: 8313

SharePoint Web Services Round Robin Service Load Balancer Event: EndpointFailure

Process Name: w3wp

Process ID: 5948

AppDomain Name: /LM/W3SVC/303076968/ROOT-1-129634027688317155

AppDomain ID: 2

Service Application Uri: urn:schemas-microsoft-com:sharepoint:service:767f7d712e064db3bb8945d9d42d0e12#authority=urn:uuid:f7625c7a33924a9981c0206f8bf0054a&authority=https://applicationserver:32844/Topology/topology.svc

Active Endpoints: 3

Failed Endpoints:1

Affected Endpoint: http://applicationserver:32843/767f7d712e064db3bb8945d9d42d0e12/SearchService.svc

Error in Diagnostic Logs:

Internal server error exception: System.ServiceModel.ServerTooBusyException: The HTTP service located at http://applicationserver:32843/767f7d712e064db3bb8945d9d42d0e12/SearchService.svc is too busy.  —> System.Net.WebException: The remote server returned an error: (503) Server Unavailable.     at System.Net.HttpWebRequest.GetResponse()     at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)

Troubleshooting Steps:

I tried to access the web service that SharePoint was trying to get to via the browser and noticed that it was reporting a 503 error as reported in the Diagnostic Logs. Then I hopped onto IIS on the affected servers and looked at the SharePoint Web Services web application (IIS) in search for a corresponding Service Application GUID that matched. The application didn’t exist on either of the application servers so that would cause the EndpointFailure as reported in the Event Viewer logs. I then double checked the Services on Server page for the application servers to make sure that the search services were turned on as having these turned on would provision the web services. They were all started which threw a little confusion my way but with some hints from this forum post, I went ahead and stopped/started the Search Query and Site Settings Service. After a little while, assuming that some timer jobs had to run, the Service Application with the missing GUID showed up in IIS and the error stopped appearing in the Event logs.

Summarized Solution:

1. Go to your Services on Server page in Central Administration.

2. Stop and Start the Search Query and Site Settings Service on each affected server as reported in the Event Viewer or SharePoint Diagnostic Logs.

Tagged: Search, Troubleshooting

§ 3 Responses to SharePoint 2010 Intermittent Search Errors & Web Services Round Robin Service Load Balancer Event: EndpointFailure

• Bryan says:

  November 3, 2011 at 5:25 am

  Thanks for the info Henry, that did stop the errors.
  We have seen this problem a couple times, the first time we ended up just recreating the Search App, but then it came back about a month later.
  We do have a case open with MS, but no idea on root cause (why does the GUID dissapear). Will update if they come back with anything earth shattering.

  Reply
• K says:

  November 17, 2011 at 12:37 pm

  Bryan,

  Any update ffrom MS regarding this issue

  Thanks
  K

  Reply
• Nishant says:

  January 25, 2012 at 10:45 am

  Thank you… it did fixed my problem…. Thank god that i found this blog…!!

  Reply