Create a new array with Timesteps and multiple features, e.g for LSTM

One trick would be to append last L-1 rows off the array and append those to the start of the array. Then, it would be a simple case of using the very efficient NumPy strides. For people wondering about the cost of this trick, as we will see later on through the timing tests, it's as good as nothing.

The trick leading upto the final goal that would support both forward and backward striding in codes would look something like this -

Backward striding :

def strided_axis0_backward(inArr, L = 2):
    # INPUTS :
    # a : Input array
    # L : Length along rows to be cut to create per subarray

    # Append the last row to the start. It just helps in keeping a view output.
    a = np.vstack(( inArr[-L+1:], inArr ))

    # Store shape and strides info
    m,n = a.shape
    s0,s1 = a.strides

    # Length of 3D output array along its axis=0
    nd0 = m - L + 1

    strided = np.lib.stride_tricks.as_strided    
    return strided(a[L-1:], shape=(nd0,L,n), strides=(s0,-s0,s1))

Forward striding :

def strided_axis0_forward(inArr, L = 2):
    # INPUTS :
    # a : Input array
    # L : Length along rows to be cut to create per subarray

    # Append the last row to the start. It just helps in keeping a view output.
    a = np.vstack(( inArr , inArr[:L-1] ))

    # Store shape and strides info
    m,n = a.shape
    s0,s1 = a.strides

    # Length of 3D output array along its axis=0
    nd0 = m - L + 1

    strided = np.lib.stride_tricks.as_strided    
    return strided(a[:L-1], shape=(nd0,L,n), strides=(s0,s0,s1))

Sample run -

In [42]: inArr
Out[42]: 
array([[1, 2],
       [3, 4],
       [5, 6]])

In [43]: strided_axis0_backward(inArr, 2)
Out[43]: 
array([[[1, 2],
        [5, 6]],

       [[3, 4],
        [1, 2]],

       [[5, 6],
        [3, 4]]])

In [44]: strided_axis0_forward(inArr, 2)
Out[44]: 
array([[[1, 2],
        [3, 4]],

       [[3, 4],
        [5, 6]],

       [[5, 6],
        [1, 2]]])

Runtime test -

In [53]: inArr = np.random.randint(0,9,(1000,10))

In [54]: %timeit make_timesteps(inArr, 2)
    ...: %timeit strided_axis0_forward(inArr, 2)
    ...: %timeit strided_axis0_backward(inArr, 2)
    ...: 
10 loops, best of 3: 33.9 ms per loop
100000 loops, best of 3: 12.1 µs per loop
100000 loops, best of 3: 12.2 µs per loop

In [55]: %timeit make_timesteps(inArr, 10)
    ...: %timeit strided_axis0_forward(inArr, 10)
    ...: %timeit strided_axis0_backward(inArr, 10)
    ...: 
1 loops, best of 3: 152 ms per loop
100000 loops, best of 3: 12 µs per loop
100000 loops, best of 3: 12.1 µs per loop

In [56]: 152000/12.1  # Speedup figure
Out[56]: 12561.98347107438

The timings of strided_axis0 stays the same even as we increase the length of subarrays in the output. That just goes to show us the massive benefit with strides and of course the crazy speedups too over the original loopy version.

As promised at the start, here's the timings on stacking cost with np.vstack -

In [417]: inArr = np.random.randint(0,9,(1000,10))

In [418]: L = 10

In [419]: %timeit np.vstack(( inArr[-L+1:], inArr ))
100000 loops, best of 3: 5.41 µs per loop

The timings support the idea of stacking to be a pretty efficient one.