subprocess popen.communicate() vs. stdin.write() and stdout.read()

Your program without communicate() deadlocks because both processes are waiting on each other to write something before they write anything more themselves.

communicate() does not deadlock in your example because it closes the stream, like the command a.stdin.close() would. This sends an EOF to your subprocess, letting it know that there is no more input coming, so it can close itself, which in turn closes its output, so a.stdout.read() eventually returns an EOF (empty string).

There is no special signal that your main process will receive from your subprocess to let you know that it is done writing the results from one command, but is ready for another command.

This means that to communicate back and forth with one subprocess like you're trying to, you must read the exact number of lines that the subprocess sends. Like you saw, if you try to read too many lines, you deadlock. You might be able to use what you know, such as the command you sent it, and the output you have seen so far, to figure out exactly how many lines to read.

You can use threads to write and read at the same time, especially if the output only needs to be printed to the user:

from threading import Thread

def print_remaining(stream):
    for line in stream:
        print(line.decode("utf-8"))

con = a.stdout.readline()
if "FATAL ERROR" not in con.decode("utf-8"):
    Thread(target=print_remaining, args=[a.stdout]).start()
    for cmd in LIST_OF_COMMANDS_TO_SEND:
        a.stdin.write(cmd)