The Python Oracle

Python Multiprocessing Lib Error (AttributeError: __exit__)

This video explains
Python Multiprocessing Lib Error (AttributeError: __exit__)

--

Become part of the top 3% of the developers by applying to Toptal
https://topt.al/25cXVn

--

Track title: CC F Haydns String Quartet No 53 in D

--

Chapters
00:00 Question
01:06 Accepted answer (Score 63)
02:34 Answer 2 (Score 1)
03:01 Thank you

--

Full question
https://stackoverflow.com/questions/2596...

Accepted answer links:
[Python 3 ]: https://docs.python.org/3/library/multip...
[contextlib.closing()]: https://docs.python.org/2/library/contex...

Answer 2 links:
[Context Manager Types]: https://docs.python.org/2/library/stdtyp...

--

Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...

--

Tags
#python #multiprocessing #pickle #withstatement #contextmanager

#avk47



ACCEPTED ANSWER

Score 65


In Python 2.x and 3.0, 3.1 and 3.2, multiprocessing.Pool() objects are not context managers. You cannot use them in a with statement. Only in Python 3.3 and up can you use them as such. From the Python 3 multiprocessing.Pool() documentation:

New in version 3.3: Pool objects now support the context management protocol – see Context Manager Types. __enter__() returns the pool object, and __exit__() calls terminate().

For earlier Python versions, you could use contextlib.closing(), but take into account this'll call pool.close(), not pool.terminate(). Terminate manually in that case:

from contextlib import closing

with closing(Pool(processes=2)) as pool:
    pool.map(myFunction, mylist)
    pool.map(myfunction2, mylist2)
    pool.terminate()

or create your own terminating() context manager:

from contextlib import contextmanager

@contextmanager
def terminating(thing):
    try:
        yield thing
    finally:
        thing.terminate()

with terminating(Pool(processes=2)) as pool:
    pool.map(myFunction, mylist)
    pool.map(myfunction2, mylist2)



ANSWER 2

Score 1


with statement is for object that have __enter__ and __exit__ functions, i.e. Context Manager Types
multiprocessing.Pool is not Context Manager Type. try do the following:

pool = Pool(processes=2)
pool.map(myFunction, mylist)
pool.map(myfunction2, mylist2)