Wavelet-based Reflection Symmetry Detection via Textural and Color Histograms

Symmetry is one of the significant visual properties inside an image plane, to identify the geometrically balanced structures through real-world objects. Existing symmetry detection methods rely on descriptors of the local image features and their neighborhood behavior, resulting incomplete symmetrical axis candidates to discover the mirror similarities on a global scale. In this paper, we propose a new reflection symmetry detection scheme, based on a reliable edge-based feature extraction using Log-Gabor filters, plus an efficient voting scheme parameterized by their corresponding textural and color neighborhood information. Experimental evaluation on four single-case and three multiple-case symmetry detection datasets validates the superior achievement of the proposed work to find global symmetries inside an image.


Introduction
Reflection symmetry is a geometrical property in natural and man-made scenes which attracts the human attention by achieving the state of equilibrium through the appearance of the similar visual patterns around [2,29]. It can be defined as a balanced region where local patterns are nearly matching on each side of a symmetry axis. This paper focuses on detecting the global mirror symmetries inside an image plane by highlighting the inter-correlation between the edge, textural and color information of major involved objects.
Many detection algorithms of reflection symmetry has been introduced in the last decade. Loy and Eklundh [17] proposed the baseline algorithm by analyzing the bilateral symmetry from image features' constellation. Firstly, lo-cal feature points (i.e. SIFT) are extracted associated with local geometrical properties and descriptor vectors. Secondly, the process of pairwise matching is computed based on the measure of the local symmetry magnitude of their descriptors. Finally, A Hough-like voting space is constructed, respect to the accumulation of symmetry candidates' magnitude, parametrized with orientation and displacement components, to select the dominant axes inside an image. Park et al. [23] presented a survey for symmetry detection algorithms, followed by two symmetry detection challenges [25,16], in which Loy and Eklundh [17] have the best symmetrical results against the participated methods [21,14,19,24]. Later, other feature-based approaches [7,5] introduced some improvements over the baseline algorithm [17]. Edge-based features [20,9,28,3,8,4,10] were more recently proposed instead of the feature points, due to their robust behavior in detecting an accurate symmetric information inside an image plane.
Log-Gabor filter was introduced by Field [11] in late 1980's, as an alternative to the Gabor filter (formerly used in [10]), to suppress the effect of DC component through the computation of the multi-scale logarithmic function in the frequency domain. This process extracts more accurate feature (edge/corner) information from an image, plus ensuring the robustness of these features with respect to illumination variations. Feature extraction methods based on Log-Gabor filter have been successfully used in different computer vision applications (biometric authentication [13,1], image retrieval [26], face recognition [15], image enhancement [27], character segmentation [18], and edge detection [12]).
Our contribution is twofold. Firstly, we extract edge features upon the application of Log-Gabor filters for symmetry detection instead of using Gabor filters. Secondly, we propose a similarity measure based on color image infor-  Figure 1: The general framework of the proposed reflection symmetry detection, using multi-scale edge detection followed by symmetry triangulation based on local geometrical, spatial and color information. mation, to improve the symmetry magnitude estimation in the voting step. In addition, we evaluate the proposed methods over all public datasets for reflection symmetry (single and multiple cases), in comparison with state-of-the-art algorithms. Figure 1 shows our proposed algorithm, which detects globally the symmetry axes inside an image plane. We firstly extract edge features using Log-Gabor filters with different scales and orientations. Afterwards, we use the edge characteristics associated with the textural and color information as symmetrical weights for voting triangulation. In the end, we construct a polar-based voting histogram based on the accumulation of the symmetry contribution (local texture and color information), in order to find the maximum peaks presenting as candidates of the primary symmetry axes.
The remaining part of the paper is organized as follows. In section 2, we explain Log-Gabor transform and its feature-based application on a gray-scale image. In section 3, textural and color histograms of window-based features are described in details. In section 4, we illustrate the detection of symmetry candidates through triangulation and voting processes. Experimental details and results are presented in section 5. Finally, section 6 contains conclusion and future work.

Log-Gabor Edge Detection
Log-Gabor filter consists of logarithmic transformation of a Gabor filter in the Fourier domain, which suppresses the negative effect of the DC component: where (η, α) are the log-polar coordinates representing radial and angular components over S scales and O orientations, associated with the frequency centers (η s , α o ) and their bandwidths (σ η , σ α ).Ĝ s (η) is multiplied by low-pass Butterworth filter U (η) of order 15, and frequency 0.45, to eliminate any extra frequencies at Fourier corners.
whereÎ GS is the gray-scale version of the image I in frequency domain, and F T (.), F T −1 (.) are the Fourier transform and its inverse. Figure 2a shows Log-Gabor wavelet filter bank with 4 scales and 5 orientations where schematic contours cover the frequency space in the Fourier domain. Consequently, the elongation scheme of Log-Gabor wavelets appears in figures 2b, 2c presenting the real and imaginary components in the spatial domain. Figure 3 presents an example of Log-Gabor response I s,o on some natural image with a blurring background. Amplitude map J(x, y) = max s,o I s,o (x, y) highlights the edge details of the foreground object in a sharp way, accompanied by precise angular values in the corresponding orientation map φ(x, y). Upon a spatial sampling of the input image I using non-interleaved cells along a regular grid (stride and cell size are proportional to the maximum image dimension max(W, H)). A feature point p i is extracted within each non-homogeneous cell D i using the wavelet response of Log-Gabor filter I s,o (D i ), associated with its maximum wavelet response J i = J(p i ) along side with the corresponding orientation φ i and color in HSV color space ψ i .

Textural and Color Histograms
The textural and color information around an edge segment are prominent similarity characteristics for natural images, describing the symmetrical behavior of local edge orientation, and the balanced distribution of luminance and chrominance components. Hence, we introduce two histograms: Firstly, neighboring textural histogram h i of size N : where 1 is the indicator function. h i is l1 normalized and circular shifted with respect to the orientation of the maximum magnitude φ i among the neighborhood cell D i . Secondly, the local HSV histogram g i of size C with subsampling rate (C hu : i is the neighborhood window around feature point p i , ψ c is a sub-sampled set of HSV space, in terms of three components: hue (hu), saturation (sa) and value (va). l1 normalization is applied to g i (.) for bin-wise histogram comparison.

Symmetry Triangulation and Voting
A set of feature pairs (p i , p j ) of size P (P −1) 2 are elected such that i = j, and P is the number of feature points. Then, we compute the symmetry candidate axis based a triangulation process with respect to the image origin. This candidate axis is parametrized by angle θ i,j (orientation of the norm), and displacement ρ i,j (norm distance to the image origin) and has a symmetry weight ω i,j defined as follows: where τ i is a direction vector associated with angle φ i , R(T ⊥ ij ) is the reflection matrix with respect to the perpendicular of the line connecting (p i , p j ) [9,10], andh j is the reverse version of h j histogram. l1 normalization is applied to symmetry weights ω. A symmetry histogram H(ρ, θ) is defined as the sum of the symmetry weights of all pairs of feature points such as: where δ is the Kronecker delta. the voting histogram H(ρ, θ) is smoothed using a Gaussian kernel to output a proper density representation (as shown in figure 4), in which the major symmetry peaks are selected by reaching-out clear extreme spots using wellknown non-maximal suppression technique [6]. The spatial boundaries of each symmetry axis is computed through the convex hull of the associated voting pairs.

Results and Discussions
This section describes the details of public symmetry datasets, evaluation metrics, and experimental settings for performance comparison of the proposed work. Performance analysis against the state-of-the-art algorithms is also provided in this section.

Evaluation metrics
The conditions represent angular and distance constraints between detection and groundtruth axes. These constraints are upper-bounded by the corresponding thresholds γ and ζ, which are defined in table 1. Furthermore, the precision P R and recall RC rates are defined by selecting the symmetry peaks according to the candidates' amplitude normalized by the highest detection score, to show the performance curve for each algorithm: where F P is a false positive (non-matched detection), and F N is a false negative (non-matched groundtruth). In addition, we used the maximum F 1 score identifying a unique comparison measure, to show the overall accuracy of each symmetry algorithm:

Experimental settings
We compare the proposed methods (Lg: without color information and LgC: with color information) against three state-of-the-art approaches: Loy and Eklundh (Loy2006) [17], Cicconet et al. (Cic2014) [9], and Elawady et al. (Ela2016) [10]). Their open-source codes are used with default parameter values for performance comparison. In Log-Gabor edge detection, we set the number of scales S and number of orientations O to 12 and 32. We also set the radial bandwidth σ η to 0.55 and the angular bandwidth σ α to 0.2. In textural and color histogram calculations, we define the number of bins for textural N and color C to 32 and 32 (sampling rate C hu : C sa : C va = 8 : 2 : 2) respectively. In case of gray-scale images, contrast values are used instead of color information in HSV color space. In symmetry voting, we declare the 2D histogram space of ρ = √ W 2 + H 2 displacement bins and θ = 360 orientation bins for extrema selection. In our experimental evaluation, the algorithms are executed to detect and compare the global symmetries inside synthetic and real-world images. Tables 2, 3 show the different versions of true positive rates for the proposed methods (Lg and LgC) against Loy and Eklundh (Loy) [17], Ci- Table 3: Using evaluation metrics ICCV17, comparison of the true positive rates based on top detection in single-case and all detections in multiple-case for the proposed methods against the state-of-art algorithms. Symmetry datasets are presented as: single-case (first 4 rows) and multiple-case (last 3 rows), highlighted between (parenthesis) the number of groundtruth for each dataset. Top 1st and 2nd values are in Bold and underlined respectively.

Datasets
Loy [17] Cic [9] Ela [10] Lg LgC  [10]. LgC performs the best result among most cases in single and multiple symmetry, due to the importance of color information for the voting computations in colorful images. At the same time, Lg has the top 2nd result, and sometimes the top 1st results in gray-scale or low-saturated images. Ela [10] ranked as the top 3rd result, due to the utilization of small grids to compute window-based features. Thanks for the advantage of SIFT features, Loy [17] is still strong competent to be ranked as the top 4th result in general. Cic [9] has the lowest performance. Figure 5 presents performance results in terms of precision and recall curves for single-case and multiplecase symmetry datasets, plus values of the maximum F 1 score to measure the performance of the proposed algorithms (Lg2017 and LgHSV2017) against Loy and Eklundh (Loy2006) [17], Cicconet et al. (Cic2014) [9], and Elawady et al. (Ela2016) [10]. In single-case symmetry, our method Lg2017 outperforms the other concurrent algorithms (Loy2006, Cic2014, and Ela2016) in the context of using only gray-scale version of involved images. Furthermore, color version of our method LgHSV2017 exploits slightly improvement over gray-scale one Lg2017. On the other hand, Only LgHSV2017 has better precision performance among others in (PSUm and NYm) datasets, due to many local symmetry groundtruth presenting inside multiple-case symmetry.
As a summary of the previous quantitative evaluations,  (2) three multiple-case symmetry (e,f,g) datasets to show the overall superior performance of our proposed methods ( Lg2017 and LgHSV2017) against the three prior algorithms (Loy2006 [17], Cic2014 [9], and Ela2016 [10]). The maximum F1 scores are qualitatively presented as square symbols along the curves, and quantitatively indicated between parentheses inside the top-right legends. Best seen on screen (zoom-in for details).
hand, the non-color proposed method Lg satisfies the singlesymmetry groundtruth in the two examples as first and fifth detections respectively. In contrast, Ela [10] mismatches the provided groundtruth with all detection and Loy [17] only finds the local symmetry axes in the object parts having same contrast values. In multiple-case images (3rd, 4th and 5th columns), LgC clearly detects the global and most of local symmetries. In the opposite side, the other methods struggle determining only more than one correct groundtruth.

Conclusion
In this paper, we detect global symmetry axes inside an image using the edge characteristics of Log-Gabor wavelet response. For the purpose of improving our results, we additionally use textural and color histograms as local symmetrical measure around edge features. We show that the proposed methods provide a great improvement over sin-  Figure 6: Some single-case and multiple-case results from AVA [10], PSUm [25,16], and NYm [8] datasets with groundtruth (blue) in 1st row (a-e). our methods in 2nd and 3rd rows (f-o) produces better results among El2016 [10] in 4rd row (p-t) and Loy2006 [17] in 5th row (u-y). For each algorithm, the top five symmetry results is presented in such order: red, yellow, green, blue, and magenta. Each symmetry axis is shown in a straight line with squared endpoints. Best seen on screen.
gle and multiple symmetry cases in all public datasets. Future work will focus on improving the voting representations, respect to the number of symmetry candidate pairs, resulting a precise selection of symmetrical axis peaks and their corresponding voting features. For art and aesthetic ranking applications, a unified balance measure is re-quired to describe the global symmetry degree inside an image.