Element | Training Statistics |
---|---|
0 | Minimum Cross-Entropy at the optimum. |
1 | Total number of Stage I iterations. |
2 | Minimum Cross-Entropy after Stage I training. |
3 | Total number of Stage II iterations. |
4 | Minimum Cross-Entropy after Stage II training. |
5 | Classification error rate from optimum network. |
Value | Enumeration | Description |
---|---|---|
0 | IMSLS_NN_NETWORK | No initialization method will be performed. Weights in NN_Network structure network will be used instead. |
1 | IMSLS_EQUAL | Equal weights |
2 | IMSLS_RANDOM | Random Weights |
3 | IMSLS_PRINCIPAL_COMPONENTS | Principal Component Weights |
4 | IMSLS_DISCRIMINANT | Discriminant Analysis Weights |
PRO t_mlff_classification_trainer_ex1
n_patterns =48 ; # of training patterns.
n_inputs =9 ; 2 nominal (7 classes) and 2 continuous.
n_nominal =7 ; 2 attributes with 3 and 4 classes each.
n_continuous =2 ; 2 continuous input attributes.
n_outputs =1 ; binary classification.
classification = LONARR(48)
nominalAtt = LONARR(n_patterns,n_nominal)
n_cat = 2
nomTempIn = LONARR(48)
inputData = $ ; size = 5*n_patterns
[0.00, 0.00, 0, 0, 0, 0.02, 0.02, 0, 1, 0, 0.04,$
0.04, 0, 2, 0, 0.06, 0.06, 0, 3, 0, 0.08, 0.08, $
1, 0, 0, 0.10, 0.10, 1, 1, 0, 0.12, 0.12, 1, 2, $
0, 0.14, 0.14, 1, 3, 0, 0.16, 0.16, 2, 0, 0, $
0.18, 0.18, 2, 1, 0, 0.20, 0.20, 2, 2, 0, 0.22, $
0.22, 2, 3, 0, 0.24, 0.28, 0, 0, 0, 0.26, 0.30, $
0, 1, 0, 0.28, 0.32, 0, 2, 0, 0.30, 0.34, 0, 3, $
0, 0.32, 0.36, 1, 0, 0, 0.34, 0.38, 1, 1, 0, 0.36,$
0.40, 1, 2, 0, 0.38, 0.42, 1, 3, 0, 0.40, 0.44, $
2, 0, 0, 0.42, 0.46, 2, 1, 0, 0.44, 0.48, 2, 2, $
0, 0.46, 0.50, 2, 3, 0, 0.52, 0.48, 0, 0, 0, 0.54,$
0.50, 0, 1, 1, 0.56, 0.52, 0, 2, 1, 0.58, 0.54, 0,$
3, 1, 0.60, 0.56, 1, 0, 1, 0.62, 0.58, 1, 1, 1, $
0.64, 0.60, 1, 2, 1, 0.66, 0.62, 1, 3, 1, 0.68, $
0.64, 2, 0, 0, 0.70, 0.66, 2, 1, 0, 0.72, 0.68, $
2, 2, 0, 0.74, 0.70, 2, 3, 0, 0.76, 0.76, 0, 0, $
1, 0.78, 0.78, 0, 1, 1, 0.80, 0.80, 0, 2, 1, 0.82,$
0.82, 0, 3, 1, 0.84, 0.84, 1, 0, 1, 0.86, 0.86, $
1, 1, 1, 0.88, 0.88, 1, 2, 1, 0.90, 0.90, 1, 3, $
1, 0.92, 0.92, 2, 0, 0, 0.94, 0.94, 2, 1, 0, 0.96,$
0.96, 2, 2, 0, 0.98, 0.98, 2, 3, 0]
contAtt = FLTARR(n_patterns,n_continuous)
colLabels = ["Pattern", "Class=0", "Class=1"]
PRINT,"***********************************"
PRINT,"* BINARY CLASSIFICATION EXAMPLE *"
PRINT,"***********************************"
; Set up continuous input attributes and
; classification target arrays.
FOR i=0L, n_patterns-1 DO BEGIN
; Assign input to array for continuous input attributes.
contAtt(i,0) = inputData(i*5)
contAtt(i,1) = inputData(i*5+1)
; Assign input to classification target array.
classification(i) = LONG(inputData(i*5+4))
ENDFOR
; Set up nominal input attributes using binary encoding.
m=0
FOR i=0L, n_cat-1 DO BEGIN
FOR j=0L, n_patterns-1 DO BEGIN
nomTempIn(j) = LONG(inputData(j*5+n_continuous+i)+1)
ENDFOR
nomTempOut= UNSUPERVISED_NOMINAL_FILTER(nomTempIn,$
N_classes=nClass)
FOR k=0L, nClass-1 DO BEGIN
FOR j=0L, n_patterns-1 DO BEGIN
nominalAtt(j,m) = nomTempOut(j,k)
ENDFOR
m = m+1
ENDFOR
ENDFOR
PRINT," TRAINING PATTERNS"
PRINT,"Y N1 N2 Z1 Z2"
FOR i=0L, n_patterns-1 DO BEGIN
j = LONG(inputData(i*5+2))
k = LONG(inputData(i*5+3))
PRINT,STRTRIM(classification(i),2),$
" ",STRTRIM(j,2),$
" ",STRTRIM(k,2),$
" ",STRING(contAtt(i,0),Format="(f4.2)"),$
" ",STRING(contAtt(i,1),Format="(f4.2)")
ENDFOR
network = MLFF_NETWORK_INIT(n_inputs, n_outputs)
network = MLFF_NETWORK(network,Create_hidden_layer=3)
network = MLFF_NETWORK(network,Create_hidden_layer=2)
network = MLFF_NETWORK(network,/Link_all)
; Note the following statement is for repeatable output.
RANDOMOPT,Set=5555l
; Train the classification network.
PRINT,"STARTING NETWORK TRAINING"
trainStats = MLFF_CLASSIFICATION_TRAINER(network,$
classification, nominalAtt, contAtt,$
/Print,Predicted_prob=classProb)
; Print class predictions.
PRINT,"Predicted Classification Probabilities"
PRINT," Pattern Class=0"
FOR i=0L, n_patterns-1 DO BEGIN
PRINT,i+1," ", STRING(classprob(i),Format="(F3.1)")
ENDFOR
END
***********************************
* BINARY CLASSIFICATION EXAMPLE *
***********************************
TRAINING PATTERNS
Y N1 N2 Z1 Z2
0 0 0 0.00 0.00
0 0 1 0.02 0.02
0 0 2 0.04 0.04
0 0 3 0.06 0.06
0 1 0 0.08 0.08
0 1 1 0.10 0.10
0 1 2 0.12 0.12
0 1 3 0.14 0.14
0 2 0 0.16 0.16
0 2 1 0.18 0.18
0 2 2 0.20 0.20
0 2 3 0.22 0.22
0 0 0 0.24 0.28
0 0 1 0.26 0.30
0 0 2 0.28 0.32
0 0 3 0.30 0.34
0 1 0 0.32 0.36
0 1 1 0.34 0.38
0 1 2 0.36 0.40
0 1 3 0.38 0.42
0 2 0 0.40 0.44
0 2 1 0.42 0.46
0 2 2 0.44 0.48
0 2 3 0.46 0.50
0 0 0 0.52 0.48
1 0 1 0.54 0.50
1 0 2 0.56 0.52
1 0 3 0.58 0.54
1 1 0 0.60 0.56
1 1 1 0.62 0.58
1 1 2 0.64 0.60
1 1 3 0.66 0.62
0 2 0 0.68 0.64
0 2 1 0.70 0.66
0 2 2 0.72 0.68
0 2 3 0.74 0.70
1 0 0 0.76 0.76
1 0 1 0.78 0.78
1 0 2 0.80 0.80
1 0 3 0.82 0.82
1 1 0 0.84 0.84
1 1 1 0.86 0.86
1 1 2 0.88 0.88
1 1 3 0.90 0.90
0 2 0 0.92 0.92
0 2 1 0.94 0.94
0 2 2 0.96 0.96
0 2 3 0.98 0.98
STARTING NETWORK TRAINING
TRAINING PARAMETERS:
Stage II Opt. = 1
n_epochs = 15
epoch_size = 48
maxIterations = 1000
maxFunctionEval = 1000
maxStep = 10.000000
functionTol = 2.42218e-05
gradientTol = 0.000345267
accuracy = 0.000345267
n_inputs = 9
n_continuous = 2
n_nominal = 7
n_classes = 2
n_outputs = 1
n_patterns = 48
n_layers = 3
n_perceptrons = 6
n_weights = 41
STAGE I TRAINING STARTING
Stage I: Epoch 1 - Cross-Entropy Error = 1.2219e-05 (Iterations=50)
(CPU Min.=0.000167)
Stage I Training Converged at Epoch = 1
STAGE I FINAL CROSS-ENTROPY ERROR = 0.000012 (CPU Min.=0.000167)
OPTIMUM WEIGHTS AFTER STAGE I TRAINING:
weight[0] = -0.62051 weight[1] = 3.09276
weight[2] = 0.758899 weight[3] = 3.25391
weight[4] = 0.880689 weight[5] = 0.570943
weight[6] = 0.582033 weight[7] = 2.02618
weight[8] = 2.10231 weight[9] = 4.52622
weight[10] = 3.17279 weight[11] = -10.8302
weight[12] = -1.11139 weight[13] = 0.922095
weight[14] = -0.715357 weight[15] = -0.427637
weight[16] = 11.0583 weight[17] = 8.9387
weight[18] = 2.18456 weight[19] = 1.82989
weight[20] = -8.44236 weight[21] = -3.11275
weight[22] = -0.00365442 weight[23] = 0.626157
weight[24] = -0.229832 weight[25] = 13.5145
weight[26] = 11.8758 weight[27] = -4.80003
weight[28] = -18.7821 weight[29] = -9.04489
weight[30] = -6.97589 weight[31] = -14.9861
weight[32] = -17.6179 weight[33] = 23.4709
weight[34] = 11.2042 weight[35] = -6.75005
weight[36] = -12.9537 weight[37] = -14.0837
weight[38] = 17.7174 weight[39] = 21.5824
weight[40] = -19.7159
STAGE I TRAINING CONVERGED
STAGE I CROSS-ENTROPY ERROR = 0.000012
0 PATTERNS OUT OF 48 INCORRECTLY CLASSIFIED
GRADIENT AT THE OPTIMUM WEIGHTS
-->g[0] = 0.000000 weight[0] = -0.620510
-->g[1] = -0.000001 weight[1] = 3.092761
-->g[2] = 0.000000 weight[2] = 0.758899
-->g[3] = -0.000001 weight[3] = 3.253912
-->g[4] = 0.000000 weight[4] = 0.880689
-->g[5] = 0.000000 weight[5] = 0.570943
-->g[6] = 0.000000 weight[6] = 0.582033
-->g[7] = -0.000000 weight[7] = 2.026184
-->g[8] = -0.000000 weight[8] = 2.102315
-->g[9] = 0.000000 weight[9] = 4.526219
-->g[10] = -0.000006 weight[10] = 3.172789
-->g[11] = 0.000000 weight[11] = -10.830215
-->g[12] = -0.000006 weight[12] = -1.111387
-->g[13] = 0.000000 weight[13] = 0.922095
-->g[14] = 0.000000 weight[14] = -0.715357
-->g[15] = 0.000000 weight[15] = -0.427637
-->g[16] = -0.000003 weight[16] = 11.058254
-->g[17] = -0.000003 weight[17] = 8.938703
-->g[18] = 0.000000 weight[18] = 2.184557
-->g[19] = -0.000005 weight[19] = 1.829893
-->g[20] = 0.000000 weight[20] = -8.442363
-->g[21] = -0.000005 weight[21] = -3.112754
-->g[22] = 0.000000 weight[22] = -0.003654
-->g[23] = 0.000000 weight[23] = 0.626157
-->g[24] = 0.000000 weight[24] = -0.229832
-->g[25] = -0.000003 weight[25] = 13.514451
-->g[26] = -0.000003 weight[26] = 11.875845
-->g[27] = 0.000001 weight[27] = -4.800026
-->g[28] = 0.000000 weight[28] = -18.782066
-->g[29] = 0.000000 weight[29] = -9.044890
-->g[30] = 0.000001 weight[30] = -6.975893
-->g[31] = 0.000001 weight[31] = -14.986099
-->g[32] = 0.000000 weight[32] = -17.617901
-->g[33] = -0.000012 weight[33] = 23.470867
-->g[34] = -0.000012 weight[34] = 11.204187
-->g[35] = -0.000001 weight[35] = -6.750047
-->g[36] = -0.000006 weight[36] = -12.953682
-->g[37] = -0.000005 weight[37] = -14.083694
-->g[38] = 0.000001 weight[38] = 17.717358
-->g[39] = 0.000001 weight[39] = 21.582405
-->g[40] = -0.000011 weight[40] = -19.715902
Training Completed - leaving training engine (CPU Min.=0.000167)
Predicted Classification Probabilities
Pattern Class=0
1 1.0
2 1.0
3 1.0
4 1.0
5 1.0
6 1.0
7 1.0
8 1.0
9 1.0
10 1.0
11 1.0
12 1.0
13 1.0
14 1.0
15 1.0
16 1.0
17 1.0
18 1.0
19 1.0
20 1.0
21 1.0
22 1.0
23 1.0
24 1.0
25 1.0
26 0.0
27 0.0
28 0.0
29 0.0
30 0.0
31 0.0
32 0.0
33 1.0
34 1.0
35 1.0
36 1.0
37 0.0
38 0.0
39 0.0
40 0.0
41 0.0
42 0.0
43 0.0
44 0.0
45 1.0
46 1.0
47 1.0
48 1.0
*** WARNING Error IMSLS_UNBOUNDED from imsls_d_mlff_classification_trainer.
*** Five consecutive steps of length "max_step" have been taken;
*** either the function is unbounded below, or has a finite
*** asymptote in some direction or the maximum allowable step
*** size "max_step" is too small.
STAGE I TRAINING STARTING
Stage I: Epoch 1 - Cross-Entropy Error = 5.50552 (Iterations=40) (CPU Min.=0.000260)
Stage I: Epoch 2 - Cross-Entropy Error = 5.65875 (Iterations=69) (CPU Min.=0.000260)
Stage I: Epoch 3 - Cross-Entropy Error = 4.83886 (Iterations=81) (CPU Min.=0.000260)
Stage I: Epoch 4 - Cross-Entropy Error = 5.94979 (Iterations=108) (CPU Min.=0.000521)
Stage I: Epoch 5 - Cross-Entropy Error = 5.54461 (Iterations=47) (CPU Min.=0.000260)
Stage I: Epoch 6 - Cross-Entropy Error = 6.04163 (Iterations=51) (CPU Min.=0.000260)
Stage I: Epoch 7 - Cross-Entropy Error = 5.95148 (Iterations=151) (CPU Min.=0.000521)
Stage I: Epoch 8 - Cross-Entropy Error = 5.5646 (Iterations=55) (CPU Min.=0.000260)
Stage I: Epoch 9 - Cross-Entropy Error = 5.94914 (Iterations=442) (CPU Min.=0.001563)
Stage I: Epoch 10 - Cross-Entropy Error = 5.94381 (Iterations=271) (CPU Min.=0.001302)
Stage I: Epoch 11 - Cross-Entropy Error = 5.41955 (Iterations=35) (CPU Min.=0.000000)
Stage I: Epoch 12 - Cross-Entropy Error = 6.01766 (Iterations=48) (CPU Min.=0.000260)
Stage I: Epoch 13 - Cross-Entropy Error = 4.20551 (Iterations=112) (CPU Min.=0.000521)
Stage I: Epoch 14 - Cross-Entropy Error = 5.95085 (Iterations=103) (CPU Min.=0.000260)
Stage I: Epoch 15 - Cross-Entropy Error = 5.9596 (Iterations=55) (CPU Min.=0.000260)
Stage I: Epoch 16 - Cross-Entropy Error = 5.96131 (Iterations=59) (CPU Min.=0.000260)
Stage I: Epoch 17 - Cross-Entropy Error = 4.83231 (Iterations=74) (CPU Min.=0.000260)
Stage I: Epoch 18 - Cross-Entropy Error = 17.1345 (Iterations=30) (CPU Min.=0.000260)
Stage I: Epoch 19 - Cross-Entropy Error = 5.95569 (Iterations=92) (CPU Min.=0.000260)
Stage I: Epoch 20 - Cross-Entropy Error = 3.15336 (Iterations=46) (CPU Min.=0.000260)
GRADIENT AT THE OPTIMUM WEIGHTS
-->g[0] = 0.675632 weight[0] = 0.075861
-->g[1] = -0.953480 weight[1] = -0.078585
-->g[2] = 1.065184 weight[2] = 2.841074
-->g[3] = 0.535531 weight[3] = -0.941049
-->g[4] = -0.019011 weight[4] = -10.638772
-->g[5] = 0.001459 weight[5] = -14.573394
-->g[6] = -0.031098 weight[6] = 6.037813
-->g[7] = -0.035305 weight[7] = 72.382073
-->g[8] = 0.011015 weight[8] = -73.564433
-->g[9] = 0.000000 weight[9] = -14.853988
-->g[10] = -0.074332 weight[10] = 2.057743
-->g[11] = 0.000522 weight[11] = -39.952435
-->g[12] = 0.063316 weight[12] = 73.164141
-->g[13] = -0.000522 weight[13] = 57.065975
-->g[14] = 1.279914 weight[14] = -0.661036
-->g[15] = -0.043097 weight[15] = -61.171894
-->g[16] = 0.003227 weight[16] = 24.236722
-->g[17] = -0.108146 weight[17] = 14.968424
-->g[18] = 0.104919 weight[18] = -39.079343
PRO mlff_classification_trainer_ex2
; Define the IMSLS_* constants.
@CMAST_COMMON
n_patterns = 150
n_inputs = 4 ; all continuous inputs
n_nominal = 0 ; no nominal
n_continuous = 4
n_outputs = 3
act_fcn = [1, 1, 1]
classification = LONARR(n_patterns)
unscaledX = FLTARR(n_patterns)
scaledX = FLTARR(n_patterns)
contAtt = FLTARR(n_patterns,n_continuous)
mean = FLTARR(n_continuous)
s = FLTARR(n_continuous)
PRINT,"*****************************************************"
PRINT,"* IRIS CLASSIFICATION EXAMPLE *"
PRINT,"*****************************************************"
; irisData(): The raw data matrix. This is a 2-D matrix with
; 150 rows and 5 columns. The last 4 columns are
; the continuous input attributes and the 1st
; column is the classification category
; (1-3). These data contain no nominal input
; attributes.
irisData = STATDATA(3)
; Set up the continuous attribute input array, contAtt(), and
; the network target classification array, classification(),
; using the above raw data matrix.
classification(*) = LONG(irisData(*,0)-1)
contAtt(*,*) = irisData(*,1:4)
; Scale continuous input attributes using z-score method.
FOR j=0L, n_continuous-1 DO BEGIN
unscaledX(*) = contAtt(*,j)
scaledX = SCALE_FILTER(unscaledX, 2,$
Return_center_spread=centerspread)
contAtt(*,j) = scaledX(*)
mean(j) = centerspread(0)
s(j) = centerspread(1)
ENDFOR
PRINT,"Scale Parameters: "
FOR j=0L, n_continuous-1 DO BEGIN
PRINT,"Var ",STRTRIM(j+1,2),$
" Mean = ",STRING(mean(j),Format="(f10.5)"),$
" S = ",STRING(s(j),Format="(f10.5)")
ENDFOR
network = MLFF_NETWORK_INIT(n_inputs, n_outputs)
network = MLFF_NETWORK(network, Create_hidden=2)
network = MLFF_NETWORK(network, /Link_all)
; Note the following statement is for repeatable output.
RANDOMOPT,Set=5555
; Train classification network.
startTime = SYSTIME(1)
trainStats = MLFF_CLASSIFICATION_TRAINER(network,$
classification, $
nominalAtt, $
contAtt, $
/Print, $
StageI=[20L, 150L], $
Init_weights_Method=IMSLS_PRINCIPAL_COMPONENTS, $
Max_step=1000.0, $
Predicted_class=predicted_class, $
Predicted_prob=predicted_class_prob,$
Class_error=class_error)
endTime = SYSTIME(1)
PRINT,"================================================="
PRINT,"Minimum Cross-Entropy Error: ",$
STRING(trainStats(0),Format="(E8.5)")
PRINT,"Classification Error Rate: ", $
STRING(trainStats(5),Format="(f8.5)")
PRINT,"Execution Time (Sec.): ",$
STRING((endTime-startTime),Format="(f8.5)")
hdr = "Predicted Class"
hdr1 = "Class | P(0) P(1) P(2) | Error"
line= "------------------------------------------"
PRINT,hdr
PRINT,line
FOR i=0L, n_patterns-1 DO BEGIN
PRINT,STRTRIM(predicted_class(i),2),$
" | ",$
STRING(predicted_class_prob(i,0),Format="(f8.5)"),$
STRING(predicted_class_prob(i,1),Format="(f8.5)"),$
;" | ",$
STRING(predicted_class_prob(i,2),Format="(f8.5)"),$
" | ",$
STRING(class_error(i),Format="(f8.5)")
if i EQ 49 OR i EQ 99 THEN BEGIN
PRINT,hdr
PRINT,hdr1
PRINT,line
ENDIF
ENDFOR
END
*******************************************************
* IRIS CLASSIFICATION EXAMPLE *
*******************************************************
% SCALE_FILTER: Alert: STAT_NORMAL_UNDERFLOW
The normal distribution is used for large degrees of freedom. However, it has produced underflow. Therefore, the probability is set to zero.
% SCALE_FILTER: Alert: STAT_NORMAL_UNDERFLOW
The normal distribution is used for large degrees of freedom. However, it has produced underflow. Therefore, the probability is set to zero.
% SCALE_FILTER: Alert: STAT_NORMAL_UNDERFLOW
The normal distribution is used for large degrees of freedom. However, it has produced underflow. Therefore, the probability is set to zero.
% SCALE_FILTER: Alert: STAT_NORMAL_UNDERFLOW
The normal distribution is used for large degrees of freedom. However, it has produced underflow. Therefore, the probability is set to zero.
Scale Parameters:
Var 1 Mean = 5.84333 S = 0.82807
Var 2 Mean = 3.05733 S = 0.43587
Var 3 Mean = 3.75800 S = 1.76530
Var 4 Mean = 1.19933 S = 0.76224
TRAINING PARAMETERS:
Stage II Opt. = 1
n_epochs = 20
epoch_size = 150
maxIterations = 1000
maxFunctionEval = 1000
maxStep = 1000.000000
functionTol = 2.42218e-05
gradientTol = 0.000345267
accuracy = 0.000345267
n_inputs = 4
n_continuous = 4
n_nominal = 0
n_classes = 3
n_outputs = 3
n_patterns = 150
n_layers = 2
n_perceptrons = 5
n_weights = 19
STAGE I TRAINING STARTING
Stage I: Epoch 1 - Cross-Entropy Error = 4.92196 (Iterations=73)
(CPU Min.=0.001000)
Stage I: Epoch 2 - Cross-Entropy Error = 5.9593 (Iterations=119)
(CPU Min.=0.001333)
Stage I: Epoch 3 - Cross-Entropy Error = 5.95304 (Iterations=193)
(CPU Min.=0.002000)
Stage I: Epoch 4 - Cross-Entropy Error = 74.9248 (Iterations=30)
(CPU Min.=0.000167)
Stage I: Epoch 5 - Cross-Entropy Error = 4.92197 (Iterations=102)
(CPU Min.=0.001500)
Stage I: Epoch 6 - Cross-Entropy Error = 5.9556 (Iterations=197)
(CPU Min.=0.003500)
Stage I: Epoch 7 - Cross-Entropy Error = 4.92196 (Iterations=101)
(CPU Min.=0.001333)
Stage I: Epoch 8 - Cross-Entropy Error = 4.92197 (Iterations=90)
(CPU Min.=0.001167)
Stage I: Epoch 9 - Cross-Entropy Error = 5.02341 (Iterations=476)
(CPU Min.=0.006833)
Stage I: Epoch 10 - Cross-Entropy Error = 5.94134 (Iterations=161)
(CPU Min.=0.001167)
Stage I: Epoch 11 - Cross-Entropy Error = 4.92203 (Iterations=82)
(CPU Min.=0.001167)
Stage I: Epoch 12 - Cross-Entropy Error = 4.92198 (Iterations=88)
(CPU Min.=0.001000)
Stage I: Epoch 13 - Cross-Entropy Error = 4.92195 (Iterations=97)
(CPU Min.=0.001333)
Stage I: Epoch 14 - Cross-Entropy Error = 5.95137 (Iterations=178)
(CPU Min.=0.001167)
Stage I: Epoch 15 - Cross-Entropy Error = 5.95102 (Iterations=130)
(CPU Min.=0.001333)
Stage I: Epoch 16 - Cross-Entropy Error = 6.90229e-05 (Iterations=184)
(CPU Min.=0.002500)
Stage I Training Converged at Epoch = 16
STAGE I FINAL CROSS-ENTROPY ERROR = 0.000069 (CPU Min.=0.028500)
OPTIMUM WEIGHTS AFTER STAGE I TRAINING:
weight[0] = 0.299444 weight[1] = -0.149455
weight[2] = 4.55187 weight[3] = -1.94491
weight[4] = -13.4141 weight[5] = -474.311
weight[6] = 1202.39 weight[7] = 2515.17
weight[8] = -3152.97 weight[9] = -176.632
weight[10] = 106.797 weight[11] = -1195.65
weight[12] = 3048.21 weight[13] = 1374.08
weight[14] = 0.991334 weight[15] = -2747.78
weight[16] = 1629.43 weight[17] = 622.712
weight[18] = -2252.16
STAGE I TRAINING CONVERGED
STAGE I CROSS-ENTROPY ERROR = 0.000069
0 PATTERNS OUT OF 150 INCORRECTLY CLASSIFIED
GRADIENT AT THE OPTIMUM WEIGHTS
-->g[0] = 0.002142 weight[0] = 0.299444
-->g[1] = 0.002150 weight[1] = -0.149455
-->g[2] = 0.001187 weight[2] = 4.551874
-->g[3] = 0.000373 weight[3] = -1.944913
-->g[4] = 0.000000 weight[4] = -13.414126
-->g[5] = 0.000000 weight[5] = -474.311462
-->g[6] = 0.000000 weight[6] = 1202.392822
-->g[7] = 0.000000 weight[7] = 2515.173584
-->g[8] = 0.000001 weight[8] = -3152.970947
-->g[9] = 0.000000 weight[9] = -176.631683
-->g[10] = -0.000000 weight[10] = 106.796753
-->g[11] = 0.000000 weight[11] = -1195.651855
-->g[12] = -0.000001 weight[12] = 3048.212646
-->g[13] = 0.000000 weight[13] = 1374.076416
-->g[14] = -0.001546 weight[14] = 0.991334
-->g[15] = 0.000000 weight[15] = -2747.777588
-->g[16] = 0.000003 weight[16] = 1629.434448
-->g[17] = -0.000003 weight[17] = 622.712341
-->g[18] = -0.000000 weight[18] = -2252.163818
Training Completed - leaving training engine (CPU Min.=0.028500)
=================================================
Minimum Cross-Entropy Error: 0.00007
Classification Error Rate: 0.00000
Execution Time (Sec.): 1.74515
Predicted Class
------------------------------------------
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
0 | 1.00000 0.00000 0.00000 | 0.00000
Predicted Class
Class | P(0) P(1) P(2) | Error
------------------------------------------
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 0.99999 0.00001 | 0.00001
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 0.99999 0.00001 | 0.00001
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 0.99998 0.00002 | 0.00002
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
1 | 0.00000 1.00000 0.00000 | 0.00000
Predicted Class
Class | P(0) P(1) P(2) | Error
------------------------------------------
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00003 0.99997 | 0.00003
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000
2 | 0.00000 0.00000 1.00000 | 0.00000