Python - Threading and a While True Loop

Use a semaphore; have the working thread release it when it's finished, and block your appending thread until the worker is finished with the semaphore.

ie. in the worker, do something like self.done = threading.Semaphore() at the beginning of work, and self.done.release() when finished. In the code you noted above, instead of the busy loop, simply do self.done.acquire(); when the worker thread is finished, control will return.

Edit: I'm afraid I don't address your needed timeout value, though; this issue describes the need for a semaphore timeout in the standard library.

Use time.sleep(seconds) to create a brief pause after each iteration of the while loop to relinquish the cpu. You will have to set the time you sleep during each iteration based on how important it is that you catch the job quickly after it's complete.

Example:

time.clock()
while True:

    if len(self.output):
        yield self.output.pop(0)

    elif self.done or 15 < time.clock():
        if 15 < time.clock():
            yield "Maximum Execution Time Exceeded %s seconds" % time.clock()
            break

    time.sleep(0.01) # sleep for 10 milliseconds

You have to use a synchronization primitive here. Look here: http://docs.python.org/library/threading.html.

Event objects seem very simple and should solve your problem. You can also use a condition object or a semaphore.

I don't post an example because I've never used Event objects, and the alternatives are probably less simple.

Edit: I'm not really sure I understood your problem. If a thread can wait until some condition is statisfied, use synchronization. Otherwise the sleep() solution that someone posted will about taking too much CPU time.

use mutex module or event/semaphore