python multithreading wait till all threads finished
Become part of the top 3% of the developers by applying to Toptal https://topt.al/25cXVn
--
Music by Eric Matyas
https://www.soundimage.org
Track title: The World Wide Mind
--
Chapters
00:00 Question
01:33 Accepted answer (Score 202)
01:58 Answer 2 (Score 245)
02:15 Answer 3 (Score 61)
03:16 Answer 4 (Score 41)
03:33 Thank you
--
Full question
https://stackoverflow.com/questions/1196...
Question links:
[here]: http://www.saltycrane.com/blog/2008/09/s.../
Accepted answer links:
[join]: http://docs.python.org/library/threading...
Answer 2 links:
[Join method]: https://docs.python.org/3/library/thread...
Answer 3 links:
https://docs.python.org/3/library/concur...
[ProcessPoolExecutor]: https://docs.python.org/3/library/concur...
--
Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...
--
Tags
#python #multithreading
#avk47
--
Music by Eric Matyas
https://www.soundimage.org
Track title: The World Wide Mind
--
Chapters
00:00 Question
01:33 Accepted answer (Score 202)
01:58 Answer 2 (Score 245)
02:15 Answer 3 (Score 61)
03:16 Answer 4 (Score 41)
03:33 Thank you
--
Full question
https://stackoverflow.com/questions/1196...
Question links:
[here]: http://www.saltycrane.com/blog/2008/09/s.../
Accepted answer links:
[join]: http://docs.python.org/library/threading...
Answer 2 links:
[Join method]: https://docs.python.org/3/library/thread...
Answer 3 links:
https://docs.python.org/3/library/concur...
[ProcessPoolExecutor]: https://docs.python.org/3/library/concur...
--
Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...
--
Tags
#python #multithreading
#avk47
ANSWER 1
Score 272
Put the threads in a list, .start() each thread, and then .join() each thread:
threads = [
Thread(...),
Thread(...),
Thread(...),
]
# Start all threads.
for t in threads:
t.start()
# Wait for all threads to finish.
for t in threads:
t.join()
ACCEPTED ANSWER
Score 210
You need to use join method of Thread object in the end of the script.
t1 = Thread(target=call_script, args=(scriptA + argumentsA))
t2 = Thread(target=call_script, args=(scriptA + argumentsB))
t3 = Thread(target=call_script, args=(scriptA + argumentsC))
t1.start()
t2.start()
t3.start()
t1.join()
t2.join()
t3.join()
Thus the main thread will wait till t1, t2 and t3 finish execution.
ANSWER 3
Score 42
I prefer using list comprehension based on an input list:
inputs = [scriptA + argumentsA, scriptA + argumentsB, ...]
threads = [Thread(target=call_script, args=(i)) for i in inputs]
[t.start() for t in threads]
[t.join() for t in threads]
ANSWER 4
Score 7
You can have class something like below from which you can add 'n' number of functions or console_scripts you want to execute in parallel passion and start the execution and wait for all jobs to complete..
from multiprocessing import Process
class ProcessParallel(object):
"""
To Process the functions parallely
"""
def __init__(self, *jobs):
"""
"""
self.jobs = jobs
self.processes = []
def fork_processes(self):
"""
Creates the process objects for given function deligates
"""
for job in self.jobs:
proc = Process(target=job)
self.processes.append(proc)
def start_all(self):
"""
Starts the functions process all together.
"""
for proc in self.processes:
proc.start()
def join_all(self):
"""
Waits untill all the functions executed.
"""
for proc in self.processes:
proc.join()
def two_sum(a=2, b=2):
return a + b
def multiply(a=2, b=2):
return a * b
#How to run:
if __name__ == '__main__':
#note: two_sum, multiply can be replace with any python console scripts which
#you wanted to run parallel..
procs = ProcessParallel(two_sum, multiply)
#Add all the process in list
procs.fork_processes()
#starts process execution
procs.start_all()
#wait until all the process got executed
procs.join_all()