NumPy: consequences of using 'np.save()' with 'allow_pickle=False'
--
Music by Eric Matyas
https://www.soundimage.org
Track title: Magical Minnie Puzzles
--
Chapters
00:00 Question
01:17 Accepted answer (Score 10)
02:32 Thank you
--
Full question
https://stackoverflow.com/questions/4169...
Question links:
[here]: https://docs.scipy.org/doc/numpy/referen...
Accepted answer links:
[this article]: https://www.cs.uic.edu/~s/musings/pickle/
--
Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...
--
Tags
#python #numpy
#avk47
ACCEPTED ANSWER
Score 14
An object array is just a normal numpy array where the dtype is object; this happens if the contents of the array aren't of the normal numerical types (like int or float, etc.). We can try out saving a numpy array with objects, just to test how this works. A simple kind of object would be a dict:
>>> import numpy as np
>>> a = np.array([{x: 1} for x in range(4)])
>>> a
array([{0: 1}, {1: 1}, {2: 1}, {3: 1}], dtype=object)
>>> np.save('test.pkl', a)
Loading this back works fine:
>>> np.load('test.pkl.npy')
array([{0: 1}, {1: 1}, {2: 1}, {3: 1}], dtype=object)
The array can't be saved without using pickle, though:
>>> np.save('test.pkl', a, allow_pickle=False)
...
ValueError: Object arrays cannot be saved when allow_pickle=False
The rule of thumb for pickles is that you're safe if you're loading a pickle that you made, but you should be careful about loading pickles that you got from somewhere else. For one thing, if you don't have the same libraries (or library versions) installed that were used to make the pickle, you might not be able to load the pickle (this is what's meant by portability above). Security is another potential concern; you can read a bit about how pickles can be abused in this article, for instance.