NNLayer Class Reference

A layer in a neural network. More...

#include <nnlayer.h>

Inheritance diagram for NNLayer:

[legend]Collaboration diagram for NNLayer:


Public Types
typedef std::vector< REAL >	WVEC
	weight vector
typedef std::vector< REAL >	DVEC
	derivative vector
Public Member Functions
	NNLayer (UINT n_in=0, UINT n_unit=0)
	NNLayer (std::istream &is)
virtual const id_t &	id () const
virtual NNLayer *	create () const
	Create a new object using the default constructor.
virtual NNLayer *	clone () const
	Create a new object by replicating itself.
UINT	size () const
void	set_weight_range (REAL min, REAL max)
const WVEC &	weight () const
void	set_weight (const WVEC &)
const WVEC &	gradient () const
void	clear_gradient ()
virtual void	initialize ()
	Initialize the model for training.
virtual REAL	train ()
	Train with preset data set and sample weight.
virtual Output	operator() (const Input &x) const
void	feed_forward (const Input &, Output &) const
void	back_propagate (const Input &, const DVEC &, DVEC &)
Protected Member Functions
virtual REAL	sigmoid (REAL) const
virtual REAL	sigmoid_deriv (REAL) const
virtual bool	serialize (std::ostream &, ver_list &) const
virtual bool	unserialize (std::istream &, ver_list &, const id_t &=empty_id)
Protected Attributes
REAL	w_min
REAL	w_max
WVEC	w
	weights and thresholds
WVEC	dw
	deravatives: w -= lr * dw
DVEC	sig_der

Detailed Description

A layer in a neural network.

This class simulates a layer of neurons.

Here's usage information.

Here's some details.

Say, we have n neurons and m inputs. The output from the neuron is

$y_i=\theta(s_i),\quad s_i=\sum_{j=0}^m w_{ij}x_j,$

where $s_i$ is the weighted sum of inputs, $x_0\equiv1$ , and $w_{i0}$ is sort of the threshold. feed_forward() does this calculation and saves $\theta'(s_i)$ for future use.

The ``chain rule'' for back-propagation says the derative w.r.t. the input x can be calculated from that to the output y. Define

$\delta_i \stackrel\Delta= \frac{\partial E}{\partial s_i} = \frac{\partial E}{\partial y_i}\frac{\partial y_i}{\partial s_i} = \frac{\partial E}{\partial y_i}\theta'(s_i).$

We have

$\frac{\partial E}{\partial w_{ij}} = \frac{\partial E}{\partial s_i} \frac{\partial s_i}{\partial w_{ij}} = \delta_i x_j,\quad \frac{\partial E}{\partial x_j} = \sum_{i=1}^n \frac{\partial E}{\partial s_i} \frac{\partial s_i}{\partial x_j} = \sum_{i=1}^n \delta_i w_{ij}.$

These equations consitute the essense of back_propagate().

Modifying sigmoid() (which computes $\theta$ ) and sigmoid_deriv() (which computes $\theta'$ ) is usually enough to get a different type of layer.

Todo:: documentation

Definition at line 53 of file nnlayer.h.

Member Typedef Documentation

typedef std::vector<REAL> DVEC

derivative vector

Definition at line 58 of file nnlayer.h.

typedef std::vector<REAL> WVEC

weight vector

Definition at line 57 of file nnlayer.h.

Constructor & Destructor Documentation

NNLayer ( UINT n_in = 0,

UINT n_unit = 0

) [explicit]

Definition at line 16 of file nnlayer.cpp.
References quick_tanh_setup().
Referenced by NNLayer::clone(), and NNLayer::create().

NNLayer ( std::istream & is ) [inline, explicit]

Definition at line 68 of file nnlayer.h.

Member Function Documentation

void back_propagate ( const Input & ,

const DVEC & ,

DVEC &

)

Definition at line 93 of file nnlayer.cpp.
References LearnModel::_n_in, LearnModel::_n_out, NNLayer::dw, LearnModel::n_input(), LearnModel::n_output(), NNLayer::sig_der, and NNLayer::w.

void clear_gradient ( )

Definition at line 58 of file nnlayer.cpp.
References NNLayer::dw.
Referenced by NNLayer::initialize().

virtual NNLayer* clone ( ) const [inline, virtual]

Create a new object by replicating itself.

Returns:
A pointer to the new copy.
The code for a derived class Derived is always
return new Derived(*this);
Though seemingly redundant, it helps to copy an object without knowing the real type of the object.
See also:
C++ FAQ Lite 20.6

Implements LearnModel.
Definition at line 72 of file nnlayer.h.
References NNLayer::NNLayer().
Referenced by FeedForwardNN::add_top().

virtual NNLayer* create ( ) const [inline, virtual]

Create a new object using the default constructor.
The code for a derived class Derived is always
return new Derived();

Implements LearnModel.
Definition at line 71 of file nnlayer.h.
References NNLayer::NNLayer().

void feed_forward ( const Input & ,

Output &

) const

Definition at line 78 of file nnlayer.cpp.
References LearnModel::_n_in, LearnModel::_n_out, lemga::op::inner_product(), LearnModel::n_input(), LearnModel::n_output(), NNLayer::sig_der, NNLayer::sigmoid(), NNLayer::sigmoid_deriv(), and NNLayer::w.
Referenced by NNLayer::operator()().

const WVEC& gradient ( ) const [inline]

Definition at line 83 of file nnlayer.h.
References NNLayer::dw.

virtual const id_t& id ( ) const [virtual]

Returns:
Class ID string (class name)

Implements Object.

void initialize ( ) [virtual]

Initialize the model for training.

Todo:
whether or when to initialize is a big problem. If we allow "continue_on_learning", then sometimes the initialization is not needed

Note:
this was a virtual function, I changed it to be empty

Reimplemented from LearnModel.
Definition at line 62 of file nnlayer.cpp.
References NNLayer::clear_gradient(), randu, NNLayer::w, NNLayer::w_max, and NNLayer::w_min.

virtual Output operator() ( const Input & x ) const [inline, virtual]

Implements LearnModel.
Definition at line 88 of file nnlayer.h.
References NNLayer::feed_forward(), and LearnModel::n_output().

bool serialize ( std::ostream & ,

ver_list &

) const [protected, virtual]

Reimplemented from LearnModel.
Definition at line 23 of file nnlayer.cpp.
References LearnModel::_n_in, LearnModel::_n_out, SERIALIZE_PARENT, NNLayer::w, NNLayer::w_max, and NNLayer::w_min.

void set_weight ( const WVEC & )

Definition at line 53 of file nnlayer.cpp.
References LearnModel::_n_in, LearnModel::_n_out, and NNLayer::w.

void set_weight_range ( REAL min,

REAL max

) [inline]

Definition at line 76 of file nnlayer.h.
References NNLayer::w_max, and NNLayer::w_min.

REAL sigmoid ( REAL ) const [protected, virtual]

Definition at line 68 of file nnlayer.cpp.
References quick_tanh().
Referenced by NNLayer::feed_forward().

REAL sigmoid_deriv ( REAL ) const [protected, virtual]

Definition at line 73 of file nnlayer.cpp.
References quick_tanh().
Referenced by NNLayer::feed_forward().

UINT size ( ) const [inline]

Definition at line 74 of file nnlayer.h.
References LearnModel::n_output().

virtual REAL train ( ) [inline, virtual]

Train with preset data set and sample weight.

Returns:
Probably the training error.

Todo:
Make the return type void

Implements LearnModel.
Definition at line 87 of file nnlayer.h.
References OBJ_FUNC_UNDEFINED.

bool unserialize ( std::istream & ,

ver_list & ,

const id_t & = empty_id

) [protected, virtual]

Reimplemented from LearnModel.
Definition at line 35 of file nnlayer.cpp.
References LearnModel::_n_in, LearnModel::_n_out, NNLayer::dw, Object::empty_id, NNLayer::sig_der, UNSERIALIZE_PARENT, NNLayer::w, NNLayer::w_max, and NNLayer::w_min.

const WVEC& weight ( ) const [inline]

Definition at line 80 of file nnlayer.h.
References NNLayer::w.

Member Data Documentation

WVEC dw [protected]

deravatives: w -= lr * dw

Definition at line 63 of file nnlayer.h.
Referenced by NNLayer::back_propagate(), NNLayer::clear_gradient(), NNLayer::gradient(), and NNLayer::unserialize().

DVEC sig_der [mutable, protected]

Definition at line 64 of file nnlayer.h.
Referenced by NNLayer::back_propagate(), NNLayer::feed_forward(), and NNLayer::unserialize().

WVEC w [protected]

weights and thresholds

Definition at line 62 of file nnlayer.h.
Referenced by NNLayer::back_propagate(), NNLayer::feed_forward(), NNLayer::initialize(), NNLayer::serialize(), NNLayer::set_weight(), NNLayer::unserialize(), and NNLayer::weight().

REAL w_max [protected]

Definition at line 61 of file nnlayer.h.
Referenced by NNLayer::initialize(), NNLayer::serialize(), NNLayer::set_weight_range(), and NNLayer::unserialize().

REAL w_min [protected]

Definition at line 61 of file nnlayer.h.
Referenced by NNLayer::initialize(), NNLayer::serialize(), NNLayer::set_weight_range(), and NNLayer::unserialize().

The documentation for this class was generated from the following files:

Generated on Mon Jan 9 23:44:57 2006 for LEMGA by

1.4.6

NNLayer Class Reference

Public Types

Public Member Functions

Protected Member Functions

Protected Attributes

Detailed Description

Member Typedef Documentation

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation