Neural Network TransferFunctions

posted Jan 20, 2016, 7:05 AM by Leroy Dyer   [ updated Jan 20, 2016, 7:25 AM ]
the neural network is a tool used for classification and has currently become a hot topic as it has accomplished some advanced classification issues, yet it can be said that a neural net is only as good as its transfer functions!!

A few transfer functions been created in this class which i have created. 

Some of the functions have derivatives and some do not.


The class itself doesn't need to be instantiated the EvaluateTransferFunct function allows for the input to return the result to the caller. 

Code

 Enum TransferFunctionType
        none
        sigmoid
        HyperbolTangent
        BinaryThreshold
        RectifiedLinear
        Logistic
        StochasticBinary
        Gaussian
        Signum
    End Enum


This Enum / Type needs to be created as it holds the names of the transfer function types needed by the class. 


I have found that this method of creating a type list enables for the select case statement to become and effective tool and a type of decision tree process can be created. 

The file which has been uploaded is the class file i have added the exstension .FILE to the end which needs to be removed from the file name to be added to the vb project of your own!


There are many more functions in the class including Gaussian, Rectified and more. 

This class can be extended for better gradient descent models and stochastic methods .




Sigmoid


    Public Shared Function Sigmoid(ByRef Value As Integer) As Double

        'z = Bias + (Input*Weight)
        'Output = 1/1+e**z
        Return 1 / (1 + Math.Exp(-Value))
    End Function


    Public Shared Function SigmoidDerivitive(ByRef Value As Integer) As Double


        Return Sigmoid(Value) * (1 - Sigmoid(Value))
    End Function


TanH

    Public Shared Function HyperbolicTangent(ByRef Value As Double) As Double
        '    TanH(x) = (Math.Exp(x) - Math.Exp(-x)) / (Math.Exp(x) + Math.Exp(-x))

        Return Math.Tanh(Value)
    End Function


    Public Shared Function HyperbolicTangentDerivative(ByRef Value As Double) As Double
        HyperbolicTangentDerivative = 1 - (HyperbolicTangent(Value) * HyperbolicTangent(Value)) * Value
    End Function


ċ
ClassTransferFunctions.VB.FILE
(6k)
Leroy Dyer,
Jan 20, 2016, 7:17 AM
v.1
ċ
ClassTransferFunctions.zip
(2k)
Leroy Dyer,
Jan 20, 2016, 7:23 AM
v.1
Comments