The cost or loss function

The following cost function compares the expected result y_ with the present result produced by the network.

cost = tf.reduce_mean(( (y_ * tf.log(Output)) + ((1 - y_) * tf.log(1.0 - Output)) ) * -1)

The tf.log is applied to the results to squash them for efficient comparisons. The tf.reduce_mean supports a variety of cost functions; this is one way to calculate the error between the training dataset and the intermediate result.

Another way to calculate a cost function, as shown in the following code, is to compare the output of the current iteration with the target output and square both to obtain a visible difference.

cost = tf.reduce_mean(tf.square(y_-Output))

Choosing a cost function will depend on your model. This method in its minimized format is called least squares. It produces efficient results in this model.