This short note is about dealing with “Connection pending send bytes is too large” error while using a Catch-up subscription and connecting to an EventStore database.
That error message can appear in the logs when the server drops a client connection. The reason why that happen is most likely that the server is sending too much data for the client to handle before the memory buffer is filled up. That’s it, the client is not handling the data at the pace the server is sending them.
Imagine to see a pipe connecting client and server. The size of the pipe is determined by the following client subscription options:
- MaxLiveQueue (default 10000)
- ReadBatchSize (default 500)
Unfortunately the default settings are way more than a client can handle in a Cloud environment where network performances can vary.
A Cloud optimised tuning I usually start with when I configure the client connection is:
- MaxLiveQueue: 2000
- ReadBatchSize: 20
An example of how to set this up in C#:
var settings = new CatchUpSubscriptionSettings(2000, 20, false, true); var conn = EventStoreConnection.Create(settings, "tcp://localhost:1113", "test-conn");
This is not the only reason why client connections can become problematic in a Cloud environment and the other most common reason are the HearthbeatInterval and HearthbeatTimeout. These two also need to be tuned as the default settings are for a local network. When I deploy in a cloud environment a client program connecting with EventStore I usually set the HearthbeatInterval=8sec and HearthbeatTimeout=3sec. Remember to set the same client timeout settings on both client and server configuration.
The combination of the subscription and connection tunings will provide your client program more stability. The frequency of client connection dropping should be greatly reduced.