Please use this identifier to cite or link to this item: http://hdl.handle.net/1893/27713
Appears in Collections:Computing Science and Mathematics Conference Papers and Proceedings
Author(s): Connor, Richard
Moss, Robert
Contact Email: richard.connor@stir.ac.uk
Title: A multivariate correlation distance for vector spaces
Editor(s): Navarro, Gonzalo
Pestov, Vladimir
Citation: Connor R & Moss R (2012) A multivariate correlation distance for vector spaces. In: Navarro G & Pestov V (eds.) Similarity Search and Applications: 5th International Conference, SISAP 2012, Toronto, ON, Canada, August 9-10, 2012. Proceedings. Lecture Notes in Computer Science, 7404. Similarity Search and Applications: 5th International Conference, SISAP 2012, Toronto, 09.08.2012-10.08.2012. Berlin, Heidelberg: Springer Verlag, pp. 209-225. https://doi.org/10.1007/978-3-642-32153-5_15
Issue Date: 31-Dec-2012
Series/Report no.: Lecture Notes in Computer Science, 7404
Conference Name: Similarity Search and Applications: 5th International Conference, SISAP 2012
Conference Dates: 2012-08-09 - 2012-08-10
Conference Location: Toronto
Abstract: We investigate a distance metric, previously defined for the measurement of structured data, in the more general context of vector spaces. The metric has a basis in information theory and assesses the distance between two vectors in terms of their relative information content. The resulting metric gives an outcome based on the dimensional correlation, rather than magnitude, of the input vectors, in a manner similar to Cosine Distance. In this paper the metric is defined, and assessed, in comparison with Cosine Distance, for its major properties: semantics, properties for use within similarity search, and evaluation efficiency. We find that it is fairly well correlated with Cosine Distance in dense spaces, but its semantics are in some cases preferable. In a sparse space, it significantly outperforms Cosine Distance over TREC data and queries, the only large collection for which we have a human-ratified ground truth. This result is backed up by another experiment over movielens data. In dense Cartesian spaces it has better properties for use with similarity indices than either Cosine or Euclidean Distance. In its definitional form it is very expensive to evaluate for high-dimensional sparse vectors; to counter this, we show an algebraic rewrite which allows its evaluation to be performed more efficiently. Overall, when a multivariate correlation metric is required over positive vectors, SED seems to be a better choice than Cosine Distance in many circumstances.
Status: VoR - Version of Record
Rights: The publisher does not allow this work to be made publicly available in this Repository. Please use the Request a Copy feature at the foot of the Repository record to request a copy directly from the author. You can only request a copy if you wish to use this work for your own research or private study.

Files in This Item:
File Description SizeFormat 
Connor Moss 2012.pdfFulltext - Published Version873.04 kBAdobe PDFUnder Permanent Embargo    Request a copy

Note: If any of the files in this item are currently embargoed, you can request a copy directly from the author by clicking the padlock icon above. However, this facility is dependent on the depositor still being contactable at their original email address.



This item is protected by original copyright



Items in the Repository are protected by copyright, with all rights reserved, unless otherwise indicated.

If you believe that any material held in STORRE infringes copyright, please contact library@stir.ac.uk providing details and we will remove the Work from public display in STORRE and investigate your claim.