new DynamicDictionaries(trie, token_info_dictionary, connection_costs, unknown_dictionary)
Dictionaries container for Tokenizer
Parameters:
| Name | Type | Description |
|---|---|---|
trie |
DoubleArray | |
token_info_dictionary |
TokenInfoDictionary | |
connection_costs |
ConnectionCosts | |
unknown_dictionary |
UnknownDictionary |