Pass kwargs to starmap while using Pool in Python
Become part of the top 3% of the developers by applying to Toptal https://topt.al/25cXVn
--
Music by Eric Matyas
https://www.soundimage.org
Track title: Magic Ocean Looping
--
Chapters
00:00 Question
01:03 Accepted answer (Score 15)
01:34 Thank you
--
Full question
https://stackoverflow.com/questions/4571...
--
Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...
--
Tags
#python #multithreading #threadpool #pythonmultiprocessing
#avk47
--
Music by Eric Matyas
https://www.soundimage.org
Track title: Magic Ocean Looping
--
Chapters
00:00 Question
01:03 Accepted answer (Score 15)
01:34 Thank you
--
Full question
https://stackoverflow.com/questions/4571...
--
Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...
--
Tags
#python #multithreading #threadpool #pythonmultiprocessing
#avk47
ACCEPTED ANSWER
Score 23
You can create a wrapper around pool.starmap that also accepts an iterator of over kwargs dictionaries.
from itertools import repeat
def starmap_with_kwargs(pool, fn, args_iter, kwargs_iter):
args_for_starmap = zip(repeat(fn), args_iter, kwargs_iter)
return pool.starmap(apply_args_and_kwargs, args_for_starmap)
def apply_args_and_kwargs(fn, args, kwargs):
return fn(*args, **kwargs)
Then you can call it in your case as:
args_iter = zip(repeat(project_name), api_extensions)
kwargs_iter = repeat(dict(payload={'a': 1}, key=True))
branches = starmap_with_kwargs(pool, fetch_api, args_iter, kwargs_iter)