Reversed cumulative sum of a column in pandas.DataFrame
This video explains
Reversed cumulative sum of a column in pandas.DataFrame
--
Become part of the top 3% of the developers by applying to Toptal
https://topt.al/25cXVn
--
Music by Eric Matyas
https://www.soundimage.org
Track title: Realization
--
Chapters
00:00 Question
01:13 Accepted answer (Score 41)
02:20 Answer 2 (Score 11)
02:35 Answer 3 (Score 1)
02:49 Answer 4 (Score 0)
03:15 Thank you
--
Full question
https://stackoverflow.com/questions/3787...
Accepted answer links:
[IPython]: https://ipython.org/
--
Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...
--
Tags
#python #pandas #dataframe #reverse
#avk47
Reversed cumulative sum of a column in pandas.DataFrame
--
Become part of the top 3% of the developers by applying to Toptal
https://topt.al/25cXVn
--
Music by Eric Matyas
https://www.soundimage.org
Track title: Realization
--
Chapters
00:00 Question
01:13 Accepted answer (Score 41)
02:20 Answer 2 (Score 11)
02:35 Answer 3 (Score 1)
02:49 Answer 4 (Score 0)
03:15 Thank you
--
Full question
https://stackoverflow.com/questions/3787...
Accepted answer links:
[IPython]: https://ipython.org/
--
Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...
--
Tags
#python #pandas #dataframe #reverse
#avk47
ACCEPTED ANSWER
Score 47
Reverse column A, take the cumsum, then reverse again:
df['C'] = df.loc[::-1, 'A'].cumsum()[::-1]
import pandas as pd
df = pd.DataFrame(
{'A': [False, True, False, False, False, True, False, True],
'B': [0.03771, 0.315414, 0.33248, 0.445505, 0.580156, 0.741551, 0.796944, 0.817563],},
index=[6, 2, 4, 7, 3, 1, 5, 0])
df['C'] = df.loc[::-1, 'A'].cumsum()[::-1]
print(df)
yields
A B C
6 False 0.037710 3
2 True 0.315414 3
4 False 0.332480 2
7 False 0.445505 2
3 False 0.580156 2
1 True 0.741551 2
5 False 0.796944 1
0 True 0.817563 1
Alternatively, you could count the number of Trues in column A and subtract the (shifted) cumsum:
In [113]: df['A'].sum()-df['A'].shift(1).fillna(0).cumsum()
Out[113]:
6 3
2 3
4 2
7 2
3 2
1 2
5 1
0 1
Name: A, dtype: object
But this is significantly slower. Using IPython to perform the benchmark:
In [116]: df = pd.DataFrame({'A':np.random.randint(2, size=10**5).astype(bool)})
In [117]: %timeit df['A'].sum()-df['A'].shift(1).fillna(0).cumsum()
10 loops, best of 3: 19.8 ms per loop
In [118]: %timeit df.loc[::-1, 'A'].cumsum()[::-1]
1000 loops, best of 3: 701 µs per loop
ANSWER 2
Score 13
Similar to unutbus first suggestion, but without the deprecated ix:
df['C']=df.A[::-1].cumsum()
ANSWER 3
Score 2
If wanting to reverse cumulative sum column-wise:
(-df).cumsum(axis=1).add(1).shift(1,axis=1,fill_value=1.0)
ANSWER 4
Score 0
This works but is slow... like @unutbu answer. True resolves to 1. Fails on False, or any other value though.
df[2] = df.groupby('A').cumcount(ascending=False)+1
df[1] = np.where(df['A']==True,df[2],None)
df[1] = df[1].fillna(method='bfill').fillna(0)
del df[2]
A B 1
# 3 False 0.277557 3.0
# 7 False 0.400751 3.0
# 6 False 0.431587 3.0
# 5 False 0.481006 3.0
# 1 True 0.534364 3.0
# 2 True 0.556378 2.0
# 0 True 0.863192 1.0
# 4 False 0.916247 0.0