[experimental] Compute perplexity of a model

Compute the perplexity of a trained word2vec model with data.

perplexity(x, targets, data, layer = c("words", "documents"))

Arguments

x

a trained textmodel_wordvector object.

targets

words for which probabilities are computed.

data

a quanteda::tokens or quanteda::dfm; the probabilities of words are tested against occurrences of words in it.

layer

the layer based on which probabilities are computed.