The Python Oracle

Pass kwargs to starmap while using Pool in Python

--------------------------------------------------
Rise to the top 3% as a developer or hire one of them at Toptal: https://topt.al/25cXVn
--------------------------------------------------

Music by Eric Matyas
https://www.soundimage.org
Track title: Puzzle Game 2 Looping

--

Chapters
00:00 Pass Kwargs To Starmap While Using Pool In Python
00:59 Accepted Answer Score 21
01:29 Thank you

--

Full question
https://stackoverflow.com/questions/4571...

--

Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...

--

Tags
#python #multithreading #threadpool #pythonmultiprocessing

#avk47



ACCEPTED ANSWER

Score 23


You can create a wrapper around pool.starmap that also accepts an iterator of over kwargs dictionaries.

from itertools import repeat

def starmap_with_kwargs(pool, fn, args_iter, kwargs_iter):
    args_for_starmap = zip(repeat(fn), args_iter, kwargs_iter)
    return pool.starmap(apply_args_and_kwargs, args_for_starmap)

def apply_args_and_kwargs(fn, args, kwargs):
    return fn(*args, **kwargs)

Then you can call it in your case as:

args_iter = zip(repeat(project_name), api_extensions)
kwargs_iter = repeat(dict(payload={'a': 1}, key=True))
branches = starmap_with_kwargs(pool, fetch_api, args_iter, kwargs_iter)