The Python Oracle

Should I use Events, Semaphores, Locks, Conditions, or a combination thereof to manage safely exiting my multithreaded Python program?

--------------------------------------------------
Rise to the top 3% as a developer or hire one of them at Toptal: https://topt.al/25cXVn
--------------------------------------------------

Music by Eric Matyas
https://www.soundimage.org
Track title: Dreaming in Puzzles

--

Chapters
00:00 Should I Use Events, Semaphores, Locks, Conditions, Or A Combination Thereof To Manage Safely Exitin
02:01 Accepted Answer Score 3
03:24 Thank you

--

Full question
https://stackoverflow.com/questions/5073...

--

Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...

--

Tags
#python #multithreading #python3x #pythonmultithreading

#avk47



ACCEPTED ANSWER

Score 3


I would use an Event to signal to a thread that it should exit:

  • create an event in __init__
  • use the event's wait() in run() for sleep and for checking when to exit
  • set the event from outside to stop the thread

To handle exceptions within a thread, I would have a try/ except block around everything it does. When something is caught, store the exception (and/or any other info you need), clean up and exit the thread.

Outside, in the main thread, check for the store exceptions in all threads, if any exception is found, signal to all threads that they should exit.

To handle exceptions in the main thread (which includes also SIGINT), have a try/except block there and signal to all threads to stop.

All together, with dummy exceptions and debug prints:

import threading
import time

class MyThread(threading.Thread):
    def __init__(self):
        super().__init__()
        self.stop_requested = threading.Event()
        self.exception = None

    def run(self):
        try:
            # sleep for 1 second, or until stop is requested, stop if stop is requested
            while not self.stop_requested.wait(1):
                # do your thread thing here
                print('in thread {}'.format(self))

                # simulate a random exception:
                import random
                if random.randint(0, 50) == 42:
                    1 / 0
        except Exception as e:
            self.exception = e

        # clean up here
        print('clean up thread {}'.format(self))

    def stop(self):
        # set the event to signal stop
        self.stop_requested.set()

# create and start some threads
threads = [MyThread(), MyThread(), MyThread(), MyThread()]
for t in threads:
    t.start()

# main thread looks at the status of all threads
try:
    while True:
        for t in threads:
            if t.exception:
                # there was an error in a thread - raise it in main thread too
                # this will stop the loop
                raise t.exception
        time.sleep(0.2)

except Exception as e:
    # handle exceptions any way you like, or don't
    # This includes exceptions in main thread as well as those in other threads
    # (because of "raise t.exception" above)
    print(e)

finally:
    print('clan up everything')
    for t in threads:
        # threads will know how to clean up when stopped
        t.stop()