tions of neural networks, the theme of the book is principles rather than applica- From the perspective of pattern recognition, neural networks can be regarded. Collection of Papers and Books concerning Deep Neural Networks - CDitzel/ Deep-Learning-Literature. Neural Networks for Pattern Recognition – Statistical foundation, perspective and alternatives, Graduate course, Semester Bet /
|Language:||English, German, Arabic|
|Genre:||Politics & Laws|
|ePub File Size:||30.65 MB|
|PDF File Size:||13.45 MB|
|Distribution:||Free* [*Registration Required]|
PDF | a pattern is an entity that is can give you a name and that is represented by a set of measured properties and the relationships between. PDF | Face Recognition has been identified as one of the attracting research areas and it has drawn the attention of many researchers due to its varying. From the perspective of pattern recognition, neural networks can be regarded understanding of the basic principles of statistical pattern recognition lies at the.
The NeuCube architecture can be used to visualize the deep connectivity inside the network before, during, and after training and thereby allows for a better understanding of the learning processes.
The paper discusses advantages and limitations of the new method and concludes that it is worth exploring further on different datasets, aiming for advances in dynamic computer vision and multimodal systems that integrate visual, aural, tactile, and other kinds of information in a biologically plausible way.
Today, algorithms for image recognition are well advanced and can be found in many applications such as search engines, security systems, industrial robots, medical devices, and virtual reality. Besides the many areas of application, another reason for the fast progress in image recognition might be the vast knowledge about the human visual system.
The eye is arguably the best studied human sensory organ and the visual cortex has been the main object of interest in a large number of neuroscientific studies. Findings from vision science have inspired the development of new hardware as well as novel algorithms and computational tools. High-definition and high-speed cameras have long surpassed the capacities of the human eye in terms of spatial and temporal resolution.
On the software side though, it still proves to be a difficult task to extend the scope of present achievements in static image recognition to dynamic visual recognition of moving objects or a moving scene.
The benefit of accurate and fast dynamic visual recognition is apparent: each of the above-mentioned applications of image recognition constitutes a potential application area for dynamic visual recognition systems.
Neural Networks and Pattern Recognition
Any kind of robot that must navigate within a three-dimensional environment or perform tasks on moving objects would benefit from an accurate and fast dynamic visual system. The popular topic of self-driving cars is only one example. Other potential implementations include security systems, automated traffic prediction and tolls, monitoring of manufacturing processes, navigational tools in air and ship traffic, or diagnostic assistants for inspections or surgery.
Since the human visual system's adaptability and efficiency are still highly superior to computer systems when it comes to tasks of dynamic vision, it is natural to let biology serve as an inspiration for the development of new computational models. Previous works have used a combination of bio-inspired visual sensors and spiking neural networks for the recognition of human postures Perez-Carrasco et al. We consider these very promising approaches, though the mentioned works lack benchmarking results that make them comparable.
In the DNA-strand-displacement implementation, weight multiplication and signal restoration are both catalytic reactions. The grey circle with an arrow indicates the direction of the catalytic cycle.
Representative, but not all possible, states are shown for the pairwise-annihilation reaction. Each domain is labelled with a name, and asterisks in the names indicate sequence complementarity.
Black-filled and white-filled arrowheads indicate the forwards and backwards directions of a reaction step, respectively. Each black number indicates the identity of a seesaw node. The location and absolute value of each red number indicates the identity and relative initial concentration of a DNA species, respectively. A red number on a wire connected to a node or between two nodes indicates a free signal molecule, which can be an input or fuel strand. A red number inside a node indicates a gate molecule, which can be a weight, summation gate or restoration gate.
A red number on a wire that stops perpendicularly at two wires indicates an annihilator molecule. A negative red number inside a half node with a zigzag arrow indicates a reporter molecule. The experimental data left, same as Fig.
All fluorescence kinetics data and simulation are shown over the course of 2.
In each output trajectory plot, dotted lines indicate fluorescence kinetics data and solid lines indicate simulation. The patterns were digitized as binary images using a linear CCD scanner. The shape recognition of the blemish mark is equivalent to the detection of the boundary of the mark.
By setting a polar coordinate system and angular sampling of the boundary, the two-dimensional patterns were converted into one-dimensional distance functions and then normalized as scaling invariant sequences. The sequences were then processed by the FT, and then the WT, to generate vectors with shift invariance.
A new method of generating shift invariance using an overcomplete wavelet transform is described in Chen and Hewit Two types of wavelet, Haar and Daubechies, are used. Feature Vector Extraction. Multiscale wavelet-transformed extremal evolution contains information of the contour primitives in a multiscale manner. Lin et al. For this reason, the multiscale wavelet-transformed extremal evolution of contour orientation was used to form the feature vector for discriminating shape-dominant points.
Wavelet Function Normalization. This permits creation of the method scale invariant and reduction of the distortion resulting from normalization of the object contours.
Shape Recognition. Observing the transformed wavelet orientation in the test images, most of the significant structures can persist in three consecutive scales, sl, s2, and s3. Thus, an extremum at Sl is regarded as a feature point if it can also appear at s2 and s3.
The feature position is decided at the first scale because the location resolution is better at a small scale. Hopfield neural networks are applied on images of industrial tools in order to test the performance. Experimental results have shown that this method can achieve satisfactory recognition under noisy, occluded, and affine conditions using only three scales. Wavelet Index of Texture for Artificial Neural Network Classification of Landsat Images In remote sensing imagery, very large amounts of data are available for research programs concerning land, atmospheric, oceanic, and climate studies with the goal of developing new models and monitoring local and global changes, as well as predicting natural and unnatural phenomena, such as volcanic eruptions, earthquakes, and the effect of human activity on our planet Szu et al.
In order to analyze all of the available data and to optimize the amount of information, remote sensing imagery needs to be organized in ways that facilitate its mining, retrieval, and future analysis. It is in this context that Szu et al. Integrating Texture Information. In fact, most of the classification errors occurring at the boundary between classes is observed when using a straightforward classification method.
Szu and colleagues thought that these errors are due to the fact that in pixel-based classification methods, such as neural networks, no local spatial information is integrated in the classification decision.
Thus, their study consists of an initial step in assessing the impact of integrating local information, particularly texture information, in the decision. Co-occurrence Matrices and Wavelet Transform. In order to study aspects of texture concerned with spatial distribution and spatial dependence among local gray tones, one often calculates several cooccurrence matrices with various distances, d, and angles, Q, and the gray tone co-occurrence matrix can be defined as a matrix of relative frequencies of two gray tones separated by a given distance, d, and at a given angles, Q Szu et al.
By using the multiresolution aspect of a wavelet transform, computations of the co-occurrence matrix can be performed at different resolutions, d, while the information from different angles, Q, are integrated within the same filter.
More generally, computing texture with wavelets allows one to raise their localization properties and provides a spatial density function of the co-occurrence texture. Wavelet Transform Used and Results. If f x, y is the image to be classified, its multiresolution wavelet decomposition is defined by ': Preliminary results show that it is an encouraging exploitation track for better discriminates of Landsat imagery.
It provides very general techniques that can be applied to many tasks in nonstationary signal processing. This section presents two applications using the wavelet transform as a prefiltering technique: We applied the multiresolution analysis using the generalized Canny Deriche GCD filter as wavelet function, first to filter the original signal and second to detect the sharp variations in the filtered signal.
Comparisons with other methods, which explored the kernel estimates technique, show that the GCD filter provided better filtering performance. We briefly discuss the main filtering methods currently used in the literature and raise the interest of a time-frequency representation of a signal.
We describe the GCD filter. Then we detail how to apply this filter with a multiresolution manner in movement analysis of handwriting and drawing. An application of this wavelet prefiltering technique for face recognition concludes this section.
Filtering Techniques and Time-Frequency Representations The existing standard signal processing methods for filtering are based primarily on two approaches: The noise to be eliminated is present in highfrequency components, and the use of FIR filters such as second-order Butterworth filters leads to a noticeable reduction of this noise.
Determination of parameters such as cutoff frequency or transition band frequencies requires a frequency analysis of the noisy signal. Statistical methods include nonparametric regression techniques such as estimates established from spline functions or kernel estimates Marquardt and Mai, The statistical approach neglects the frequency representation of the signal in favor of a temporal representation of the signal.
Direct extraction of significant information duration of phases, changes in rhythm, discontinuities, etc.
If such a model cannot be a priori assumed, smoothing and differentiating the signal while considering its frequency components also become necessary. For nonperiodic signals, the Fourier frequency analysis more precisely, the integral of Fourier represents the signal as a superposition of sinusoidal waves of all possible frequencies of which the contribution is coded through their respective amplitudes.
This method is powerful for stationary signals but is limited for nonstationary signals such as speech, music signals, and handwriting signals. In this case, the wavelet transform allows decomposition of the signal into functions of both time and frequency.
Wavelet Transform and Generalized Canny Deriche Filter The wavelet transform can be applied in order to extract the characteristics of a signal presenting sharp variations. For this scope, a multiscale analysis first introduced by Mallat and Zhong is usually used. Most multiscale sharp variation detectors smooth the signal at various scales and detect sharp variation points from their first or second derivative.
The extrema of the first derivative corresponds to the inflection points of the smoothed signal. We call a smoothing function any function qS t whose integral is equal to 1 and converges to 0 at infinity. For must purposes, wavelet function is not required to keep a continuous scale parameter s.
In order to allow fast numerical implementations, we impose the condition that the scale varies along the dyadic sequence 2j with j E Z. In our application of wavelet transforms to handwriting or drawing signals and face recognition, we used multiresolution analysis using the "GCD filter" as wavelet function.
These Z transforms derive two third-order recursive filters moving in opposite directions. Details for the computing of these numerical filters are given in the Appendix. In order to illustrate multiresolution analysis, a noisy signal presenting slow and fast variations can be used. Figure 29 illustrates the advantages of the multiscale decomposition: Illustration of the multiresolution analysis.
An optimal calibration of the filter parameters was made for a synthetic signal, and the filtering performance was assessed by estimating the residual errors in the position, velocity, and acceleration signals. The GCD filter provided better performance than other currently used filtering techniques.
We also present how this technique can be applied to the problem of signal segmentation in the case of drawing movements. Handwriting and Drawing Signals As for language and perception, the study of handwriting and drawing performances has progressively become a stimulating object of research, attracting scientists from very different horizons.
For instance, neurophysiologists, experimental psychologists, and neurologists are concerned with handwriting and drawing in the context of an understanding of human movement planning, programming, and execution, including their respective disturbances.
The interests of electronic engineers and computer scientists in handwriting and drawing are related to image processing in general, such as automatic pattern recognition or signature authentification, as well as to signal processing, through, for instance, the improvement of the technical performances of digitizers or the refinement of valid techniques for the analysis of handwriting signals.
Handwriting and drawing can be studied both within a task-oriented approach, which focuses on the analysis of the products, i. The analysis of pen-tip displacements over a digitizer provides very valuable information for the study of human motor control [e. Globally, this approach requires computing different parameters from the first velocity , second acceleration , and sometimes third jerk derivatives of the pen-tip displacements as a function of time.
Different types of errors can be introduced by the very process of digitizing the pen-tip displacements. Ward and Phillips have distinguished several of them, such as missing coordinates and nonlinearity as examples of spatial errors or temporal asynchronisms in the sampling of x and y coordinates as an example of a temporal error.
As pointed out by Marquardt and Mai , when a kinematics movement analysis is desired, the greatest difficulty emerges from the random errors introducing stochastic noise in the signal because this noise will necessarily be largely amplified as successive derivatives of the signal are computed, with differentiation acting as a high-pass filter.
In addition to the performances of the digitizer, noise can emerge from different sources when human movement is recorded, such as noise caused by stretch reflexes, a physiological tremor, or mechanical oscillations due to the spring-like characteristics of limbs Van Galen et al.
Having at one's disposal highly efficient filtering methods appears crucial, and several solutions are used in the literature. Most of them deal with standard linear filter methods for digital low-pass filtering. Teulings and Maarse used a filter cutoff frequency of 10 Hz, but other parameters have been adopted in the literature [a transition band from 8 to 24 Hz in Teulings et al.
Clearly no agreement for the determination of these filter parameters is established in the literature. The main problem here is related to the representativity of the handwriting samples used for the determination of cutoff frequencies. Some solutions consist of nonparametric regression methods, such as kernel estimates, performant methods for an automatic definition of the filter parameters having been proposed Amico and Ferrigno, The kernel estimate approach has been tested directly for handwriting signals and provides better results than FIR filters or second-order Butterworth filters for the first and second derivatives Marquardt and Mai, The present section suggests that the wavelet filtering method provides an optimal solution for the movement analysis of handwriting and drawing signals.
We will show that the solution is optimal both for the filtering procedure and for the precise location of significant positions occurring in the course of movement trajectory, these allowing a segmentation of the movement into meaningful units. Choice of Optimal Parameters for Filtering.
In order to select the optimal scales for the smoothing of handwriting signals, synthetic data sets have to be studied. We therefore decided to consider the same synthetic data sets as those studied by Marquardt and Mai in order to facilitate a further comparative assessment of our method. In order to simulate hand-written loops as optimally as possible, these authors used sine waves at 3, 5, and 7 Hz, computed with an amplitude of 10 mm sampled at Hz, adding a Gaussian noise of cr x -- 0.
The determination of optimal filtering parameters is done with position, velocity, and acceleration signals. In order to study position, the original noisy signal is filtered with the smoothing GCD filter and then the positional error is measured in the filtered signal.
In order to study velocity and acceleration, the first and second derivatives of the filtered signal are computed and, subsequently, the velocity and acceleration errors are measured.
The optimal smoothing scale can thus be obtained by minimization of the different mean square errors MSE. Figure 30 displays the results, showing the relation among MSE, scale, and signal frequency. Figure 30 reveals the existence of an optimal smoothing scale with the presence of minima on the different curves.
For each class of signal, position, velocity, and acceleration, the scale that gives the best filtering result is a function of signal frequency. In order to improve the quality of filtering, the choice of the smoothing scale necessarily lies in a compromise among the three available values at 3, 5, and 7 Hz , which can be achieved by determining the best minimization of the three errors. For this scope, the following equation was computed: Comparison Tests.
Relation among MSE, scale, and frequency. We therefore computed the standard deviations of the residual errors in the filtered signals and their respective derivative for a 1-Hz sine wave sampled at Hz. Table 1 reports the obtained results. Results obtained with the GCD filter show a significant improvement of the smoothing performances in comparison to the other filters.
The GCD filter offers a particularly good performance for the acceleration signal. However, these performances are available only after a calibration of the filter parameters.
Optimal scale values depend on the sampled frequency of the noisy signal. For the sake of illustration, a Hz sampled signal should be filtered using 2, 1.
A few technical details may be useful.
The software discussed in this study is written in Matlab 5. The time for filtering equidistant data points at a sampled rate of Hz is 4.
This time is independent from the scale because we use the same third-order filters for these three signals, with different coefficients adapted to the chosen scale as described in the Appendix. This fast filtering time allows filtering of the original signal in almost real time. Because the essence of multiscale resolution analysis lies in the detection of variations in the signal, it can also be fruitfully used for resolving one of the main problems in handwriting and drawing research, that of automatic segmentation.
Several algorithms and methods have been developed for recovering meaningful pieces of information from the recorded signal, such as letters handwriting or geometrical drawing segments see Wesolkowski, ; Bontempi and Marcelli, We now briefly report how we adapted the multiresolution analysis to this question.
The method was applied to the segmentation of angular figures, made of three segments see Bontempi and Marcelli, ; Deshief et al. The scope was to resituate the three segments and the pauses that possibly occur at angles Figure 31 , which display the tangential velocity profile Figure 31B when drawing an obtuse figure Figure 31A delimited by the circles on the trace Figure 31E.
The primary interest of our method was in combining both spatial detection of the relevant local curvature maxima the angles and detection of the relevant local minima of velocity. Detecting only the velocity minima appeared too restrictive because as soon as the movement departs from ballistic-like movements, such as in children or in patients, several velocity minima are found between two relevant spatial segmentation points.
The algorithm of spatial detection was aimed at providing candidate positions for a segmentation realized further in the velocity domain, eliminating minor accidents appearing along the trace.
For our application, only positions associated with the angles had to be detected. Segmentation of angular figures. In reference to Lee et al. A gradient descent search was then carried out until the first local velocity minimum. In the case of a pause, two successive minima were obtained with this procedure.
Discussion Most of the criteria used for the assessment of handwriting and drawing movements are derived from velocity and acceleration signals. The degree of fluency of movements, for instance, is often considered as an indicator of the level of coordination in movement production Van Galen, and requires the analysis of the velocity profile, if not the acceleration profile Hogan and Flash, of the recorded signal.
Smoothing the original recorded signal with a filter resistant to the differentiation process is crucial to the validity of further analyses. In this perspective, considerable improvements in available filtering techniques have been made Woltring, ; Marquardt and Mai, As indicated clearly by residual errors measurement, when compared to other currently used filters, the best performances are offered by the wavelet transform, both for velocity and acceleration signals.
However, one must consider that these optimal performances require a calibration of the filter parameters on the basis of the sampled frequency of the recorded signal, but this limit appears as a minor drawback because the variety of sampling frequencies used by researchers in the domain is not large to Hz essentially. The very good results obtained with wavelet analysis in different applications such as speech, music, and image analysis are confirmed in the application for handwriting and drawing signals see also Lee et al.
A second part of our study was devoted to the question of segmentation of the recorded signals into meaningful segments for movement analysis. Again, the wavelet transform was used for this scope and provided interesting results, as also obtained by Lee et al. Of course, further analysis is still needed here for an assessment of the power of the method if an automatic algorithm would be desired.
To what extent such an approach can bring some solutions for the recognition of drawn or written characters is also still an open question.
Application to Face Recognition Linear autoassociative memories are one of the most simple and wellstudied neural network models Kohonen, ; Anderson et al.
Even though linear autoassociators are. One of the ways to improve performance could be to use some pre- and postprocessing on the patterns to be recognized. This section evaluates the performance of a preprocessing technique using the wavelet transform applied to face images.
In order to improve the performance of a linear autoassociator, we examined the use of several preprocessing techniques. The gist of our approach is to represent each pattern by one or several preprocessed i. We found that the multiscale Canny-Deriche operator gives the best performance of all models.
Second, we compared the performance of the multiscale CannyDeriche operator with the control condition on a pattern completion task of noise-degraded versions with several levels of noise of learned faces and new faces of the same or another race than the learned faces. In all cases, the multiscale Canny-Deriche operator performs significantly better than the control.
This section is organized as follows. First, we describe the linear autoassociator model applied to face images. Second, we compare the wavelet approach with other preprocessing or filtering techniques. Third, we look at the performance of the wavelet approach under various conditions of noise degradation. Linear Autoassociators and Eigenvalue Decomposition a. Linear Autoassociator Description. The advantage of linear associators in comparison with nonlinear models is that they provide integration of a very large number of cells in the network.
Their implementation is quite easy because they can be analyzed in terms of the singular value decomposition of a matrix Valentin et al. In addition, linear models constitute a first processing stage for numerous applications using more sophisticated approaches for reviews, see Valentin et al.
For example, Kohonen showed that an autoassociative memory could act as a content addressable memory for face images. Figure 32 gives an illustration. When linear autoassociative memories are applied to images, the first step is to transform each digitized image into a image vector by concatenating the columns of the matrix of the pixel values of the image. Images are "stored" into a connection-weight matrix, which models neural synaptic connections between neural cells associated with the image pixels see Figure In our description, we follow closely the formulation detailed in Abdi The patterns to be learned are represented by L x 1 vectors ak where k is the stimulus number.
The components of ak specify the values of the pattern to be applied to the L cells of the input layer for the kth stimulus. The responses of the network are given by L x 1 vectors ok. The complete set of K stimuli is represented by an L x K matrix noted A i. The set of K responses is represented by an L x K matrix noted O. Illustration of a content-addressable memory for faces. Images of faces were stored using an autoassociative memory.
Top Two stimuli given as a key to probe the memory. Bottom Responses of the memory. The memory is able to reconstruct a face from an incomplete input cf. Abdi, The linear autoassociator is applied to images: In order to achieve a high level of performance, several iterative learning rules have been proposed. The most popular one is clearly the Widrow-Hoff learning rule. This is an iterative procedure that corrects the connection matrix W using the difference between the target response and the actual response of the network.
Eigen and Sigular Value Decomposition: The PCA Approach. Abdi and colleagues developed a simple method of implementing the WidrowHoff algorithm by using the eigen decomposition of W or singular decomposition of matrix A. These decompositions give rise to PCA. Eigenvectors of a matrix are vectors that have the property that, when multiplied by the matrix, their length is changed but their direction remains unchanged.
Traditionally, the set of eigenvectors of a given matrix is represented by a matrix U in which the first column represents the vector with the largest eigenvalue, the second column the eigenvector with the second largest eigenvalue, and so on. The corresponding eigenvalues are represented by a diagonal matrix A.
The notions of eigenvectors and eigenvalues of a positive semidefinite matrix can be used to define the singular value decomposition SVD of a rectangular matrix. Abdi et al. Typically, N is significantly smaller than L i.
Preprocessing Using Multiscale Edges The goal of learning is to find values for the connections between cells such that the response of the model approximates the input as well as possible.
If learning has been successful, then the response pattern will be more similar to the original pattern than the degraded stimulus was for an illustration, see Kohonen, In other words, autoassociators can act as pattern completion devices. Here, we explore different approaches for improving the performance of a linear autoassociator storing face images.
The general strategy is to store, in addition to the original images, several filtered versions of the images see Figure We refer to this technique as preprocessing.
Then the model is evaluated by its reconstruction performance when presented with probes that are versions of the original faces degraded by the addition of Gaussian random noise. Because we are interested in image patterns, we choose filtering techniques meaningful in this context.
Because it is generally agreed that edges are essential for recognition Jia and Nixon, , we decided to increase their importance in the image. Quite a large number of algorithms have been proposed in the literature for performing edge extraction. We decided to implement three algorithms. The Sobel operator a differential operator is considered a standard procedure well suited for noiseless images.
The Sobel operator was implemented with a convolution and a 3 x 3 mask. The GCD operator because it is known to be optimal for edge extraction in noisy images Deriche, ; Bourennane et al. The multiscale GCD edge detector Bourennane et al. The GCD filter is a separable filter when applied to two-dimensional images. Figure 34 displays the impulse response of the generalized CannyDeriche filter for different scales.
As a result, this filter detects edges occurring at different scale resolutions in the image Mallat and Zhong, Impulse response of the generalized Canny-Deriche filter for different scales. In order to compare these different techniques, we implemented these three models along with two control conditions corresponding to a standard associator, and for denoising used a Wiener filter because it is a standard algorithm applied one image at a time.
All the models were tested using the same procedure. The patterns stored were a set of 80 face images of size x with 16 gray levels per pixel. The performance of each model was assessed as follows. Each face was degraded by adding to each pixel a random number [chosen such that the noise values belong to the interval 0 to 45 ].
The degraded face was then recalled by the model or filtered in the case of the Wiener filter model. The correlation between the original face and the model-reconstructed face reflects the performance of the model for this face: Specifically, the models were a Wiener filter applied directly to the noise-degraded stimulus and four autoassociators see Figure A standard autoassociator storing the original 80 face images.
Prefiltering for pattern recognition using wavelet transform and neural networks
An autoassociator storing the original 80 face images plus, for each face, a Sobel-filtered image of the face hence a total of face images. An autoassociator storing the original 80 face images plus, for each face, a GCD-filtered image of the face hence a total of face images. An autoassociator storing the original 80 face images plus, for each face, four wavelet-transformed by a multiscale GCD filter face images one face image per scale resolution, hence a total of images.
For the last three models, the complete set of patterns to be learned matrix A is composed of original and filtered images. The eigenvector matrix U and the synaptic connection matrix W have to be obtained using Eqs. Figure 36 displays an example of the responses of the models to the test face.
Patterns to be learned by four autoassociators: Filtered images have been obtained, respectively, with the Sorel operator, the optimized Canny-Deriche operator, and the wavelet transform. Response of the models. Top A target face learned previously by the autoassociator and the stimulus used to test the model the stimulus is a noisy version of the target. Bottom Responses of the different models. The wavelet model performed best.
Mean correlation between the model responses and their targets the higher the correlation, the better the performance. The b o t t o m panels show the estimation of the original face by a standard autoassociator, a Wiener filter, an autoassociator plus Sobel preprocessing, on autoassociator plus a G C D filter, and an autoassociator plus a wavelet transform.
Clearly, the standard method is the worst of all. Preprocessing the images improves the performance of the autoassociator, and the wavelet transform gives the best result.
In conclusion, the multiscale resolution i. Therefore, we decided, in what follows, to consider only this approach. Pattern Completion of Noisy Patterns We have applied the multiscale edge preprocessing to store a set of 80 Caucasian faces 40 males and 40 females.
In order to evaluate the effect due to preprocessing, we tested the models with different levels of Gaussian random noise added to the test stimulus. Learning was implemented as described previously. For simplicity, we decided to keep only two models: Testing was implemented as described previously except that faces were tested under four different levels of noise.
The noise intensity was chosen such that its range was, respectively, [ Figure 38 displays an example of the noisy test stimuli used along with the response of each model standard and wavelet. Top Four stimuli obtained by adding to a target stimulus a noise component of magnitude equal to one, two, three, and four times the magnitude of the signal of the original stimulus. Middle Responses produced by the standard autoassociator. Bottom The response of the autoassociator when learning is "enhanced" with wavelet-filtered stimuli.
Stimuli and responses. Top Performance for a new Caucasian face and bottom for a new Japanese face. Left to fight A noise-degraded stimulus the magnitude of the noise is equal to twice the magnitude of the original signal ; the response of the standard autoassociator; and the response of the wavelet-enhanced autoassociator. We also decided to explore the performance of the model with three different types of face stimuli: This was done in order to evaluate the robustness of the models in terms of response generalization to new stimuli.
Figure 39 displays, as an example, the responses of both models for two new faces from top to bottom: The autoassociator trained with the standard learning is not capable of producing distinguishable responses. As can be seen in Figure 39, better results are obtained with wavelet preprocessing. Figure 40 displays the mean correlation between noiseless face images and the output for each model 1 for 80 previously learned Caucasian faces, 2 for 80 new Caucasian faces, and 3 for 80 new Japanese faces.
In all cases, preprocessing the image improves the performance of the autoassociator with the improvement being more important when the noise added is larger. This section explored the effects of storing, in a linear autoassociator, filtered versions of face images in addition to the original images. The multiscale generalized Canny-Deriche operator produces better generalization performance than the control with or without noise added to the image.
Mean correlation between model response as a function of the magnitude of the noise for Caucasian faces learned previously, new Caucasian faces, and new Japanese faces. Filled lines correspond to the wavelet-enhanced model, and dotted lines correspond to the standard autoassociator. The wavelet-enhanced autoassociator always performed best. Les R6seaux de neurones. Presses Universitaires de Grenoble, Grenoble. Abdi, H. Amico, M. Comparison between the more recent techniques for smoothing and derivative assessment in biomechanics.
Anderson, J. Distinctive features, categorical perception, and probability learning: Some applications of a neural model. Arslan, L. Selective training for hidden markov models with application to speech classification. IEEE Trans. Speech Audio Process 7 1 , Ayrulu, B.
Neural networks for improved target differentiation and localization with sonar. Neural Networks 14, Bachman, G. Bartlett, M. Benedetto, J. Mathematics and Applications. Bishop, C. Press, London. On-line segmentation of cursive script using an arclength representation. In "Handwriting and Drawing Research: Basic and Applied Issues" M.
Simmer, C. Leedham, and A. Thomassen, eds.
IOS Press, Amsterdam. Bourennane, E. Amrlioration du filtre de CannyDeriche pour la drtection des contours sous forme de rampe. Traitement du signal, Recherche, Vol. Broomhead, D. Multivariable functional interpolation and adaptive networks.
Complex Syst. Chen, H. Application of wavelet transforms and neural networks to the recognition and classification of blemishes. Mechatronics 10, Chen, C. Classification of underwater signals using wavelet transform and neural networks, Math. Model 27 2 , Chui, C. Coifman, R. Wavelet analysis and signal processing. In "Wavelet and Their Applications. Entropy-based algorithms for best basis selection. Theory 38 2 , Das, M.
A bivariate autoregressive modelling technique for analysis and classification of planar shapes. Pattern Anal Mach. Daubechies, I. Orthonormal bases of compactly supported wavelets. Pure Appl. Ten lectures on wavelets. Deriche, R. Using canny's criteria to derive a recursively implemented optimal edge detector, lnt. Desbiez, D. Biomechanical and perceptual determinants of drawing angles. Acta Psychol. Drolon, H. Particles shape analysis and classification using the wavelet transform.
Pattern Recogn. Eide, A. Artificial neural networks as measuring devices. Nuclear Instruments Methods physics Res. A, Englehart, K. Classification of the myoelectric signal using timefrequency based representation. Farge, M. Wavelet transform and their applications to turbulance. Fluid Mech. Fausett, L. Fukunage, K. Gabor, D. Theory of communication. IEE 93, Grosman, D. Gurney, K. Haykin, S.The basic idea is that if two units, i and j, are active simultaneously, their interconnection must be strengthened.
The classifier used is an MLP neural network with two outputs: A new method of generating shift invariance using an overcomplete wavelet transform is described in Chen and Hewit Preprocessing and Feature Extraction.
In other words, autoassociators can act as pattern completion devices. The GCD operator because it is known to be optimal for edge extraction in noisy images Deriche, ; Bourennane et al. Revista Mathematica Ibero Americana, Vol. Units of the hidden layer transform their activation i.