python - numba @njit to update a big dict -


i try use numba function need search on big (10e6) dict (int, int) tuple key.

import numpy np numba import njit  myarray = np.array([[0, 0],  # 0, 1                     [0, 1],                     [1, 1],  # 1, 2                     [1, 2],  # 1, 3                     [2, 2],                     [1, 3]] ) # lot of shape~(10e6, 2)  dict_with_tuples_key = {(0, 1): 1,                         (3, 7): 1} # ~10e6 keys  

a simplified version this

# @njit def update_dict(dict_with_tuples_key, myarray):     line in myarray:         i, j = line         if (i, j) in dict_with_tuples_key:             dict_with_tuples_key[(i, j)] += 1         else:             dict_with_tuples_key[(i, j)] = 1     return dict_with_tuples_key  new_dict = update_dict(dict_with_tuples_key, myarray) print new_dict  new_dict = update_dict2(dict_with_tuples_key, myarray) # print new_dict # {(0, 1): 2,   # +1 in dict_with_tuples_key #  (0, 0): 1,   # diag #  (1, 1): 1,   # diag #  (2, 2): 1,   # diag #  (1, 2): 1,   # new myarray #  (1, 3): 1,   # new myarray #  (3, 7): 1 } 

it appear @njit not accept dict function arg ?

i'm wondering how rewrite this, specially if (i, j) in dict_with_tuples_key part search.

njit means function compiled in nopython mode. dict, list , tuple python objects , therefore not supported. not arguments , not inside function.

if dict keys different consider using 2d numpy array first axis represents first index of dict-key-tuple , second axis second index. rewrite as:

from numba import njit import numpy np  @njit def update_array(array, myarray):     elements = myarray.shape[0]     in range(elements):         array[myarray[i][0]][myarray[i][1]] += 1      return array   myarray = np.array([[0, 0], [0, 1], [1, 1],                     [1, 2], [2, 2], [1, 3]])  # calculate size of numpy array replaces dict: lens = np.max(myarray, axis=0) # maximum values array = np.zeros((lens[0]+1, lens[1]+1)) # create empty array hold indexes in myarray update_array(array, myarray) 

since indexed dictionary tuples transition problems indexing array not great.


Comments

Popular posts from this blog

sublimetext3 - what keyboard shortcut is to comment/uncomment for this script tag in sublime -

java - No use of nillable="0" in SOAP Webservice -

ubuntu - Laravel 5.2 quickstart guide gives Not Found Error -