python - Why does this code trying all permutations of a list element with all other members, exit with exit status 137? -

i have large corpus of text (~170kb) want train nlp application.

however, trying make list of elements of list, paired other elements of same list 1 @ time, causes program exit exit code 137.

def wordstobigrams(words):     totalsentencebigrams = [(a,b) b in words in words] 

you running out of memory when building list. avoid that, use the itertools library , process each pair generated. or save generated pairs file later.

specifically, use product function:

equivalent nested for-loops in generator expression. example, product(a, b) returns same ((x,y) x in y in b).

edit: yes, script requires large amounts of memory - memory consumption on machine: