'list' object has no attribute 'map' in pyspark

--------------------------------------------------
Rise to the top 3% as a developer or hire one of them at Toptal: https://topt.al/25cXVn
--------------------------------------------------

Music by Eric Matyas
https://www.soundimage.org
Track title: Puzzle Game 3 Looping

--

Chapters
00:00 'List' Object Has No Attribute 'Map' In Pyspark
00:29 Accepted Answer Score 6
00:56 Thank you

--

Full question
https://stackoverflow.com/questions/4709...

--

Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...

--

Tags
#python #apachespark #pyspark #bigdata

#avk47

ACCEPTED ANSWER

Score 6

map(filterOut2, data) works:

>>> data = [[1,2,3,5],[1,2,5,2],[3,5,2,8],[6,3,1,2],[5,3,2,5],[4,1,2,5] ]
... def filterOut2(line):
...     return [x for x in line if x != 2]
... list(map(filterOut2, data))
...
[[1, 3, 5], [1, 5], [3, 5, 8], [6, 3, 1], [5, 3, 5], [4, 1, 5]]

map() takes exactly 1 argument (2 given)

Looks like you redefined map. Try __builtin__.map(filterOut2, data).

Or, use a list comprehension:

>>> [filterOut2(line) for line in data]
[[1, 3, 5], [1, 5], [3, 5, 8], [6, 3, 1], [5, 3, 5], [4, 1, 5]]