Discussion:
TIdSchedulerOfThreadPool problems
(too old to reply)
woomla
2008-08-29 07:26:15 UTC
Permalink
Hi all,

I have a HTTP server which needs to handle very much requests. I want
to use TIdSchedulerOfThreadPool to avoid having a Thread created for
every request. I experience some problems though. Without setting
IdHTTPServer->Scheduler to the ThreadPool, and thus using the default,
there are no problems. But then the app creates so many Threads (>50 /
sec). With reusing, I only need less then 10 threads. So I set the
PoolSize at 10 and MaxThreads at 50.

When I am in the middle of a IdHTTPServerCommandGet (the webpage
continuously loads jpegs), and I change to another webpage, the
connection (of the current request / Context) is closed. When trying
to WriteContent, I receive a ClosedGracefully Exception. WriteContent
Terminates the Thread (Yarn is Stopped and thus the Thread is
destroyed). This itself is no problem, as soon as there are no more
threads to handle a request, a new Thread (and TIdYarnOfThread) is
created and remains in the pool (for a maximum of 10).

But here comes the trouble. When this scenario happens, only the first
few requests are handled and the other requests are not handled
anymore. It seems to me that the thread that is being terminated is
still used to handle a request and thus freezes the handling of the
other request. But here my knowledge and debugging skills for Indy
stop. When this happens once, terminating the server will hang in
freeing the Threads (Yarns). It looks like the Thread is Suspended and
it never gets a resume. If I reload the webpage after a short while,
all requests are handled correctly.

The webpage in question has many jpegs, css's and javascripts so when
requesting the page, a lot of requests are done in short time. This
requesting and it's timing is handled by the browser (FireFox in this
case). When I use a proxy (i.e. Fiddler), which slightly delays the
request, Indy can handle the requests fine. This indicates to me a
timing (and possible locking) issue.

That's it, I hope I've made myself clear and someone can help me get
past this problem (should I write my own pool?) In the mean time, I
will continue to look into this problem and see if I can pinpoint the
problem more accurate. (But I have to work with C++Builder and cannot
compile Protocols for debug, it gives an internal compiler error,
though the rest compiles fine in debug).

Wim
woomla
2008-08-29 14:22:06 UTC
Permalink
More information as I can now debug also Protocols (apart from
IdSSLOpenSLLHeaders, but I've worked around that).

When things goes wrong, I've noticed that a Yarn stays in the
ActiveYarn list (I guess this is the yarn where the exception occurs).
Subsequent requests that follows immediately stops for some reason.
When terminating the application, the Yarn in the ActiveYarn list is
not released. The Thread is Suspended according to the debugger, but
the Suspended property says it is not.

Wim
woomla
2008-09-01 12:44:18 UTC
Permalink
I believe the problem is that a terminated thread is not removed from
the ActiveYarn pool in TIdSchedularOfThread or the ThreadPool in
TIdSchedulerOfThreadPool.

Here is what could happen:

An IdThread receives an exception (10053) and terminates.
It is not destroyed because FreeOnTerminate is false.
The ThreadPool isn't aware of the thread that is stopped.
A new IdYarn for a new request is created using this thread and is put
in the ActiveYarns pool.
The new thread is started (resumed) but because it is finished, it
actually never runs, leaving the Yarn in the pool and stopping other
requests for the requested web page.

When terminating the webserver application TerminateAllYarns is
called.
The yarn with the terminated, not suspended and not destroyed thread,
is called to Stop.
The thread actually will never remove itself from the ActiveYarn pool
not when it is destroyed nor terminated.

I would really like someone to either confirm or reject this bug,
thanks in advance.

Wim
a***@gmail.com
2008-10-12 17:12:09 UTC
Permalink
Hi,

Your detective work could be very helpful for the Indy developers.
Nevertheless, all the messages posted through Google Groups (like this
one) won't never appears on the "official" Embarcadero newsgroups :
https://forums.codegear.com/forum.jspa?forumID=74

So if you need answers, get a CDN account and use the CodeGear forum
gateway instead of Google groups.

Regards,

A.R.
woomla
2008-11-10 09:14:01 UTC
Permalink
Post by a***@gmail.com
Hi,
Your detective work could be very helpful for the Indy developers.
Nevertheless, all the messages posted through Google Groups (like this
one) won't never appears on the "official" Embarcadero newsgroups :https://forums.codegear.com/forum.jspa?forumID=74
So if you need answers, get a CDN account and use the CodeGear forum
gateway instead of Google groups.
Regards,
A.R.
Thanks for your reply.

I've joined http://tech.groups.yahoo.com/group/Indy-Dev-Public and
used this to solve my problem.
I didn't knew about the codegear forum. I wonder if it is in 'sync'
with the yahoo group.

Wim

Loading...