Compute similarity/distance between rows or columns of large matrices

Fast similarity/distance computation function for large sparse matrices. You can floor small similarity value to to save computation time and storage space by an arbitrary threshold (min_simil) or rank (rank). You can specify the number of threads for parallel computing via options(proxyC.threads).

Usage

simil(
  x,
  y = NULL,
  margin = 1,
  method = c("cosine", "correlation", "jaccard", "ejaccard", "fjaccard", "dice", "edice",
    "hamann", "faith", "simple matching"),
  min_simil = NULL,
  rank = NULL,
  drop0 = FALSE,
  diag = FALSE,
  use_nan = NULL,
  sparse = TRUE,
  digits = 14
)

dist(
  x,
  y = NULL,
  margin = 1,
  method = c("euclidean", "chisquared", "kullback", "jeffreys", "jensen", "manhattan",
    "maximum", "canberra", "minkowski", "hamming"),
  p = 2,
  smooth = 0,
  drop0 = FALSE,
  diag = FALSE,
  use_nan = NULL,
  sparse = TRUE,
  digits = 14
)

Arguments

x: matrix or Matrix object. Dense matrices are covered to the CsparseMatrix-class internally.
y: if a matrix or Matrix object is provided, proximity between documents or features in x and y is computed.
margin: integer indicating margin of similarity/distance computation. 1 indicates rows or 2 indicates columns.
method: method to compute similarity or distance
min_simil: the minimum similarity value to be recorded.
rank: an integer value specifying top-n most similarity values to be recorded.
drop0: if TRUE, removes zero values to make the similarity/distance matrix sparse. It has no effect when dense = TRUE.
diag: if TRUE, only compute diagonal elements of the similarity/distance matrix; useful when comparing corresponding rows or columns of x and y.
use_nan: if TRUE, returns NaN if the standard deviation of a vector is zero when method is "correlation"; if all the values are zero in a vector when method is "cosine", "chisquared", "kullback", "jeffreys" or "jensen". Note that use of NaN makes the similarity/distance matrix denser and therefore larger in RAM. If FALSE, return zero in same use situations as above. If NULL, will also return zero but also generate a warning (default).
sparse: if TRUE, returns sparseMatrix object. When neither min_simil nor rank is used, dense matrices require less space in RAM.
digits: determines rounding of small values towards zero. Use primarily to correct floating point errors. Rounding is performed in C++ in a similar way as zapsmall.
p: weight for Minkowski distance
smooth: adds a fixed value to all the cells to avoid division by zero. Only used when method is "chisquared", "kullback", "jeffreys" or "jensen".

Details

Available Methods

Similarity:

cosine: cosine similarity
correlation: Pearson's correlation
jaccard: Jaccard coefficient
ejaccard: the real value version of jaccard
fjaccard: Fuzzy Jaccard coefficient
dice: Dice coefficient
edice: the real value version of dice
hamann: Hamann similarity
faith: Faith similarity
simple matching: the percentage of common elements

Distance:

euclidean: Euclidean distance
chisquared: chi-squared distance
kullback: Kullback–Leibler divergence
jeffreys: Jeffreys divergence
jensen: Jensen–Shannon divergence
manhattan: Manhattan distance
maximum: the largest difference between values
canberra: Canberra distance
minkowski: Minkowski distance
hamming: Hamming distance

See the vignette for how the similarity and distance are computed: vignette("measures", package = "proxyC")

Parallel Computing

It performs parallel computing using Intel oneAPI Threads Building Blocks. The number of threads for parallel computing should be specified via options(proxyC.threads) before calling the functions. If the value is -1, all the available threads will be used. Unless the option is used, the number of threads will be limited by the environmental variables (OMP_THREAD_LIMIT or RCPP_PARALLEL_NUM_THREADS) to comply with CRAN policy and offer backward compatibility.

Examples

mt <- Matrix::rsparsematrix(100, 100, 0.01)
simil(mt, method = "cosine")[1:5, 1:5]
#> Warning: x or y has vectors with all zero; consider setting use_nan = TRUE to set these values to NaN or use_nan = FALSE to suppress this warning
#> 5 x 5 sparse Matrix of class "dsTMatrix"
#>               
#> [1,] 1 0 . 0 .
#> [2,] 0 1 . 0 .
#> [3,] . . . . .
#> [4,] 0 0 . 1 .
#> [5,] . . . . .
mt <- Matrix::rsparsematrix(100, 100, 0.01)
dist(mt, method = "euclidean")[1:5, 1:5]
#> 5 x 5 sparse Matrix of class "dsTMatrix"
#>                                                  
#> [1,] 0.000000 1.421267 2.150558 2.088636 1.213425
#> [2,] 1.421267 0.000000 1.923564 1.854077 0.740000
#> [3,] 2.150558 1.923564 0.000000 2.458150 1.775528
#> [4,] 2.088636 1.854077 2.458150 0.000000 1.700000
#> [5,] 1.213425 0.740000 1.775528 1.700000 0.000000