Approach for a multi-server application

[Felipe]

Hello!

I'm very new to POCO and would like to say that the library is very well written and easy to follow. Congratulations.

Well, I'm developing an application that needs to have multiple servers in it. Only one server will be high traffic. I have the following requirements:

Requirement 1) The application needs to support a high traffic TCP Server listening on port X.
Requirement 2) The application needs to support a very low traffic TCP Server listening on port Y.
Requirement 3) The application needs to support a very low traffic TCP SSL Server listening on port Z.
Requirement 4) The application needs to support a very simple low traffic HTTP Server.

I've came with the following solution for the requirements:

Requirement 1) Use of an instance of a TCPServer class together with an instance of a ServerSocket class bound to port X. The TCPServer instance's thread pool would be created with as many threads as the cores of the system.
Requirement 2) Use of an instance of a TCPServer class together with an instance of a ServerSocket class bound to port Y. The TCPServer instance's thread pool would be created a single thread.
Requirement 3) Use of an instance of a TCPServer class together with an instance of a SecureServerSocket class bound to port Z. The TCPServer instance's thread pool would be created a single thread.
Requirement 4) Use of an instance of a HTTPServer class together with an instance of a ServerSocket. The HTTPServer instance's thread pool would be created a single thread.

The reason why I'm putting all this together in the same application is because of communication issues. For example, once an SSL connection arrives, I'll need to stop all TCP servers, wait until the connection ends, update some values related to the TCP server and then restart them. By the way, I'm planning to use Poco::Events to help me with those communications =).

Do you think this is a reasonably approach?

Thank you very much.

[alex]

Do you think this is a reasonably approach?

It depends on the number of connections you'll be handling and response time you expect from it. We do not have a Proactor pattern implemented, but if it is applicable to your case or especially you are on a system that hass (e)poll call available, you may want to go with Reactor for the high-volume server (we do have a multi-threaded Reactor in the release pipeline, but it is not released yet). For the single threaded ones, if you do not want to use a thread per each request, you may consider avoiding dependency on ThreadPool (which may be busy enough servicing your high volume stuff) by writing your own single-threaded TCP server (e.g see test EchoServer).
But, really, to give you a proper advice, one would need more details (and time).

[Felipe]

But, really, to give you a proper advice, one would need more details (and time).

Ah, sorry for the lack of information.

The server will be running on Linux (probably a Debian or Ubuntu Server). It will run in a controlled environment, so we can limit the number of connections. It is already in production, but it's kinda legacy and makes use of pure BSD sockets, so we are looking for a higher abstraction level alternative.

We want to maximize throughput, and this could involve keeping connections alive. But, the way clients use it right now is by opening a connection, requesting the service and closing the connection. We can change this, if necessary, but if it could work well in both ways would be better.

Not sure if I'm asking for too much, I'm never good at network programming =p

By the way, thanks for your time.

[Felipe]

Hey there! I've ended up doing what I've thought at first, and worked well. Since just using the frameworks have the big benefit of simplicity, I've just adjusted the size of each thread pool, and was enough for my use cases.

I've used Poco::Util:ServerApplication for my main class which initializes all servers which are instances of Poco::Util::Subsystem. For the communication between the subsystems, I'm using the Notification scheme, registered in the method initialize inherited from Poco::Util::Application.

What I've loved about Poco is that the source code is very easy to follow, with tons of useful commentaries. I didn't have any problem understanding (quickly) how the notification works and how concurrency is treated there.

I have one last question. I've understood that the TCPServer framework enqueue connections before they are picked by connection threads (which create the TCPConnection object and call start()). Is this queue guaranties that the connections are queued by the order they arrive? And, if so, is there any easy way to get this order?

Thank you.

[alex]

I have one last question. I've understood that the TCPServer framework enqueue connections before they are picked by connection threads (which create the TCPConnection object and call start()). Is this queue guaranties that the connections are queued by the order they arrive?

yes

And, if so, is there any easy way to get this order?

Depends what you mean by "easy". There is no way without modifying the library and the only option Notification queue offers (in addition to simple FIFO enqueuing) is urgent notification enqueuing (which adds notification to the front of the queue). See Poco::Net::TCPServerDispatcher and Poco::NotificationQueue for details.

[Felipe]

Depends what you mean by "easy". There is no way without modifying the library and the only option Notification queue offers (in addition to simple FIFO enqueuing) is urgent notification enqueuing (which adds notification to the front of the queue). See Poco::Net::TCPServerDispatcher and Poco::NotificationQueue for details.

Thanks, solved my problems.