Implementation of the Spark technique in a matrix distributed computing algorithm
Küçük Resim Yok
Tarih
2022
Yazarlar
Dergi Başlığı
Dergi ISSN
Cilt Başlığı
Yayıncı
Walter De Gruyter Gmbh
Erişim Hakkı
info:eu-repo/semantics/openAccess
Özet
Two analyzes of Spark engine performance strategies to implement the Spark technique in a matrix distributed computational algorithm, the multiplication of a sparse multiplication operational test model. The dimensions of the two input sparse matrices have been fixed to 30,000 x 30,000, and the density of the input matrix have been changed. The experimental results show that when the density reaches about 0.3, the original dense matrix multiplication performance can outperform the sparse-sparse matrix multiplication, which is basically consistent with the relationship between the sparse matrix multiplication implementation in the single-machine sparse matrix test and the computational performance of the local native library. When the density of the fixed sparse matrix is 0.01, the distributed density-sparse matrix multiplication outperforms the same sparsity but uses the density matrix storage, and the acceleration ratio increases from 1.88x to 5.71x with the increase in dimension. The overall performance of distributed operations is improved.
Açıklama
Anahtar Kelimeler
Spark Technology, Distributed, Matrix Operation, Sparse Matrix, Dense Matrix, Performance, Operation
Kaynak
Journal Of Intelligent Systems
WoS Q Değeri
N/A
Scopus Q Değeri
Q2
Cilt
31
Sayı
1
