pandas: filter rows of DataFrame with operator chaining
--------------------------------------------------
Hire the world's top talent on demand or became one of them at Toptal: https://topt.al/25cXVn
and get $2,000 discount on your first invoice
--------------------------------------------------
Music by Eric Matyas
https://www.soundimage.org
Track title: Horror Game Menu Looping
--
Chapters
00:00 Pandas: Filter Rows Of Dataframe With Operator Chaining
00:32 Accepted Answer Score 476
01:07 Answer 2 Score 78
01:24 Answer 3 Score 164
01:41 Answer 4 Score 42
02:05 Thank you
--
Full question
https://stackoverflow.com/questions/1186...
--
Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...
--
Tags
#python #pandas #dataframe
#avk47
Hire the world's top talent on demand or became one of them at Toptal: https://topt.al/25cXVn
and get $2,000 discount on your first invoice
--------------------------------------------------
Music by Eric Matyas
https://www.soundimage.org
Track title: Horror Game Menu Looping
--
Chapters
00:00 Pandas: Filter Rows Of Dataframe With Operator Chaining
00:32 Accepted Answer Score 476
01:07 Answer 2 Score 78
01:24 Answer 3 Score 164
01:41 Answer 4 Score 42
02:05 Thank you
--
Full question
https://stackoverflow.com/questions/1186...
--
Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...
--
Tags
#python #pandas #dataframe
#avk47
ACCEPTED ANSWER
Score 476
I'm not entirely sure what you want, and your last line of code does not help either, but anyway:
"Chained" filtering is done by "chaining" the criteria in the boolean index.
In [96]: df
Out[96]:
A B C D
a 1 4 9 1
b 4 5 0 2
c 5 5 1 0
d 1 3 9 6
In [99]: df[(df.A == 1) & (df.D == 6)]
Out[99]:
A B C D
d 1 3 9 6
If you want to chain methods, you can add your own mask method and use that one.
In [90]: def mask(df, key, value):
....: return df[df[key] == value]
....:
In [92]: pandas.DataFrame.mask = mask
In [93]: df = pandas.DataFrame(np.random.randint(0, 10, (4,4)), index=list('abcd'), columns=list('ABCD'))
In [95]: df.ix['d','A'] = df.ix['a', 'A']
In [96]: df
Out[96]:
A B C D
a 1 4 9 1
b 4 5 0 2
c 5 5 1 0
d 1 3 9 6
In [97]: df.mask('A', 1)
Out[97]:
A B C D
a 1 4 9 1
d 1 3 9 6
In [98]: df.mask('A', 1).mask('D', 6)
Out[98]:
A B C D
d 1 3 9 6
ANSWER 2
Score 164
Filters can be chained using a Pandas query:
df = pd.DataFrame(np.random.randn(30, 3), columns=['a','b','c'])
df_filtered = df.query('a > 0').query('0 < b < 2')
Filters can also be combined in a single query:
df_filtered = df.query('a > 0 and 0 < b < 2')
ANSWER 3
Score 78
The answer from @lodagro is great. I would extend it by generalizing the mask function as:
def mask(df, f):
return df[f(df)]
Then you can do stuff like:
df.mask(lambda x: x[0] < 0).mask(lambda x: x[1] > 0)
ANSWER 4
Score 42
Since version 0.18.1 the .loc method accepts a callable for selection. Together with lambda functions you can create very flexible chainable filters:
import numpy as np
import pandas as pd
df = pd.DataFrame(np.random.randint(0,100,size=(100, 4)), columns=list('ABCD'))
df.loc[lambda df: df.A == 80] # equivalent to df[df.A == 80] but chainable
df.sort_values('A').loc[lambda df: df.A > 80].loc[lambda df: df.B > df.A]
If all you're doing is filtering, you can also omit the .loc.