MK-NN: Program for MLP training and analysis

download MKNN v.0.947

The datasets must be in text files of the following format: xor and must be saved with a .dat extension.

About MK-NN

• MK-NN is a very good program expect for two little problems:
  • I was writing the program only for my own use and I have never had time to make it easy to use for others.
  • Some options don't work and some are intentionally blocked.
    
    
• The program is written in Borland Delphi 6 Personal Edition.
  
  
• Program requirements: Windows 95 or newer with minimal screen resolution 1024x768 (1280x1024 recommended).
  
  
• The simplified error surface module as a Matlab guide (for Matlab 6.x/7.x with NNToolbox 4.x and Matlab 2007 with NNTolbox 5.x) is accessible here. (Overwrite the original train*.m files with that from nnet/nnet directory for NNToolbox 4.x and with that from nnet/nnet/nntrain directory for NNToolbox 5.x) Note that mesh() doesn't always correctly connect the points.
  
  

MK-NN Features

• MLP training algorithms: VSS, NG, BP (Variable Step Search, Numerical Gradient, Backpropagation)
  
  
• Analysis of MLP trainings performed with any other application (e.g. Matlab NN Toolbox) that can save weights and errors after each epoch to a text file.
  
  
• SMLP network for logical rule extraction.
  
  
• Plots:
  • 3D MLP Error Surface projection in PCA directions
  • Particular weights values in a given point on the error surface (mouse-driven animation)
  • 2D MLP Error Surface sections in gradient or next step directions
  • 3D MLP Error Surface projection in PCA directions with learning trajectory
  • Comparison of 3D error surface error and real error (mouse-driven animation)
  • 2D MLP Learning Trajectories projection in 6 PCA directions
  • 3D MLP Learning Trajectories projection in 6 PCA directions
  • 1D Learning Trajectories projection in 6 PCA directions with trajectory extrapolation
  • User-defined modifications of angle and altitude of 3D MLP Error Surface projections
  • All the above projections in any directions specified in a text file
  • Training Progress (error, accuracy, weight vector length
  • Neuron Signals in 2D, 3D or parallel coordinates after every epoch
  • Angle between step directions in various learning algorithms
  • Gradient or next step components and the distance to error minimum in their directions
  • 2D projections of the training or test datasets
  • perhaps also something more what I have forgotten to list here
  • In preparation: various projections of MLP decision borders
    
    
• Text files:
  • network, training and dataset parameters and statistics
  • weight matrix
  • eigenvalues and eigenvectors
  • neuron signals in response to every vector of the training or test set
  • characteristic points of some plots (you can import such a file into Matlab, make the plot and save the figure in EPS format, if you want to attach it to a Tex document)
  • logical rules generated by SMLP network
    
    

How to use the program

The user's guide is obviously under construction and I am not sure if I finish it whenever.

Fig.1. The main form

There are many options in the program, which work correctly only in certain configurations. I will describe below only the most typical usage scenario:

• select training method and parameters
• to import training data from another program, select Matlab as the training method
• press New Training and select the dataset from the OpenFileDialog box
• to see the training progress, check the appropriate boxes in the Plots page
• to see the training results, from the main menu: Results and choose either on the training set or on the test set. Choose the test set from the OpenFileDialog box
• check detailed result file from the plots page
• if you want crossvalidation tests: from the main menu: Open -> NG crossvalidation or VSS crossvalidation
• to obtain error surface projection, Error Surface page -> SVD + Plot"
• to obtain learning trajectory projection, press Trajectory 2D
• to manually extrapolate the trajectory in the first six PCA direction, press Trajectory 1D, then go through all the six directions, try to imagine where the next point may lie and keeping "CTRL" press the mouse button (a red cross will appear). Then you will see on the main form the old and new error.
• to perform SMLP training for logical rule extraction, SMLP page -> SMLP Training

The labels are associated with hints - place the mouse cursor on the label to read the hint. If the program stops working correctly - delete mknn.ini. This is the file where the configuration is kept. After deleting the file, the program starts with default parameters.

Fig. 2. Error Surface page on the main form

Fig. 3. Network and Error page on the main form

Fig. 4. SMLP page on the main form

Fig. 5. Plots page on the main form

Fig. 6. Numerical Gradient page on the main form

Fig. 7. Error Surface Form

Fig. 8. Trajectory 2D form

... and many other plots.

Creative Commons License. You are free to copy, share and adapt all articles and software from my web page for noncommercial purposes, provided that you attribute the work to me and place a link to my home page. What you build upon my works may be distributed only under the same or similar license and you may not distort the meaning of my original texts.