Membership for list of arrays: ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() error problem

Essentially, you can't use in to test for numpy arrays in a Python list. It will only ever work for the first element, because of an optimisation in the way Python tests for equality.

What's happening is that the implementation for list.__contains__ (which in defers to), is using a short-cut to find a match faster, by first checking for identity. Most people using Python know this as the is operator. This is faster than == equality checks because all is has to do is see if the pointers of both objects are the same value, it is worth checking for first. An identity test works the same for any Python object, including numpy arrays.

The implementation essentially would look like this if it was written in Python:

def __contains__(self, needle):
    for elem in self:
        if needle is elem or needle == elem:
            return True
    return False

What happens for your list of numpy arrays then is this:

• for q in Q, step 1: q = Q[0]

  • q in Q is then the same as Q.__contains__(Q[0])
    • Q[0] is self[0] => True!

• for q in Q, step 2: q = Q[1]

  • q in Q is then the same as Q.__contains__(Q[1])
    • Q[1] is self[0] => False :-(
    • Q[1] == self[0] => array([False, False]), because Numpy arrays use broadcasting to compare each element in both arrays.

The array([False, False]) result is not a boolean, but if wants a boolean result, so it is passed to (C equivalent of) the bool() function. bool(array([False, False])) produces the error you see.

Or, done manually:

>>> import numpy as np
>>> Q = [np.array([0, 1]), np.array([1, 2]), np.array([2, 3]), np.array([3, 4])]
>>> Q[0] is Q[0]
True
>>> Q[1] is Q[0]
False
>>> Q[1] == Q[0]
array([False, False])
>>> bool(Q[1] == Q[0])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

You'll have to use any() and numpy.array_equal() to create a version of list.__contains__ that doesn't use (normal) == equality checks:

def list_contains_array(lst, arr):
    return any(np.array_equal(arr, elem) for elem in lst)

and you can then use that to get True for your loop:

>>> for q in Q:
...     print(list_contains_array(Q, q))
...
True
True
True
True

When you compare two Python lists for equality, you get a single True or False result:

>>> [0, 1] == [0, 1]
True
>>> [0, 1] == [1, 2]
False

But when you compare numpy arrays for equality, the result is another numpy array where each element i is the result of comparing the ith element of the first array with the ith element of the second array:

>>> np.array([0, 1]) == np.array([0, 1])
array([ True,  True])
>>> np.array([0, 1]) == np.array([1, 2])
array([False, False])

When you execute q in Q, it will test q for is with each element of Q and if they are not the same elements it will then do a comparison for equality. This is repeated until if finds an "equal" match (identity or compares equal), in which case it will return True, else return False if it finds no "equal" match.

When q is the first element of Q as is the case the first time through the loop, it compares itself first with the first element of Q and finds the is condition exists and thus returns True:

>>> True if Q[0] is Q[0] or Q[0] == Q[0] else False
True

But the second time through the loop, q is the second element of Q and when it is compared with the first element of q, since the is condition is not met, it tries to do an equality comparison:

>>> True if Q[1] is Q[0] or Q[1] == Q[0] else False
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()