lemga::NNLayer

NNLayer Class Reference

A layer in a neural network. More...

#include <nnlayer.h>

Inheritance diagram for NNLayer:

Inheritance graph
[legend]
Collaboration diagram for NNLayer:

Collaboration graph
[legend]
List of all members.

Public Types

typedef std::vector< REAL > WVEC
 weight vector
typedef std::vector< REAL > DVEC
 derivative vector

Public Member Functions

 NNLayer (UINT n_in=0, UINT n_unit=0)
 NNLayer (std::istream &is)
virtual const id_t & id () const
virtual NNLayer * create () const
 Create a new object using the default constructor.
virtual NNLayer * clone () const
 Create a new object by replicating itself.
UINT size () const
void set_weight_range (REAL min, REAL max)
const WVEC & weight () const
void set_weight (const WVEC &)
const WVEC & gradient () const
void clear_gradient ()
virtual void initialize ()
virtual void train ()
 Train with preset data set and sample weight.
virtual Output operator() (const Input &x) const
void feed_forward (const Input &, Output &) const
void back_propagate (const Input &, const DVEC &, DVEC &)

Protected Member Functions

virtual REAL sigmoid (REAL) const
virtual REAL sigmoid_deriv (REAL) const
virtual bool serialize (std::ostream &, ver_list &) const
virtual bool unserialize (std::istream &, ver_list &, const id_t &=NIL_ID)

Protected Attributes

REAL w_min
REAL w_max
WVEC w
 weights and thresholds
WVEC dw
 deravatives: w -= lr * dw
DVEC sig_der

Detailed Description

A layer in a neural network.

This class simulates a layer of neurons.

Here's usage information.

Here's some details.

Say, we have n neurons and m inputs. The output from the neuron is

\[ y_i=\theta(s_i),\quad s_i=\sum_{j=0}^m w_{ij}x_j, \]

where $s_i$ is the weighted sum of inputs, $x_0\equiv1$, and $w_{i0}$ is sort of the threshold. feed_forward() does this calculation and saves $\theta'(s_i)$ for future use.

The ``chain rule'' for back-propagation says the derative w.r.t. the input x can be calculated from that to the output y. Define

\[ \delta_i \stackrel\Delta= \frac{\partial E}{\partial s_i} = \frac{\partial E}{\partial y_i}\frac{\partial y_i}{\partial s_i} = \frac{\partial E}{\partial y_i}\theta'(s_i). \]

We have

\[ \frac{\partial E}{\partial w_{ij}} = \frac{\partial E}{\partial s_i} \frac{\partial s_i}{\partial w_{ij}} = \delta_i x_j,\quad \frac{\partial E}{\partial x_j} = \sum_{i=1}^n \frac{\partial E}{\partial s_i} \frac{\partial s_i}{\partial x_j} = \sum_{i=1}^n \delta_i w_{ij}. \]

These equations consitute the essense of back_propagate().

Modifying sigmoid() (which computes $\theta$) and sigmoid_deriv() (which computes $\theta'$) is usually enough to get a different type of layer.

Todo:
documentation

Definition at line 53 of file nnlayer.h.

Member Typedef Documentation

typedef std::vector<REAL> DVEC
 

derivative vector

Definition at line 58 of file nnlayer.h.

typedef std::vector<REAL> WVEC
 

weight vector

Definition at line 57 of file nnlayer.h.

Constructor & Destructor Documentation

NNLayer (  UINT  n_in = 0,
UINT  n_unit = 0
)  [explicit]
 

Definition at line 16 of file nnlayer.cpp.

References quick_tanh_setup().

Referenced by NNLayer::clone(), and NNLayer::create().

NNLayer (  std::istream &  is  )  [inline, explicit]
 

Definition at line 68 of file nnlayer.h.

Member Function Documentation

void back_propagate (  const Input &  ,
const DVEC &  ,
DVEC & 
) 
 

Definition at line 93 of file nnlayer.cpp.

References LearnModel::_n_in, LearnModel::_n_out, NNLayer::dw, LearnModel::n_input(), LearnModel::n_output(), NNLayer::sig_der, and NNLayer::w.

void clear_gradient (   ) 
 

Definition at line 58 of file nnlayer.cpp.

References NNLayer::dw.

Referenced by NNLayer::initialize().

virtual NNLayer* clone (   )  const [inline, virtual]
 

Create a new object by replicating itself.

Returns:
A pointer to the new copy.
The code for a derived class Derived is always 
 return new Derived(*this);
Though seemingly redundant, it helps to copy an object without knowing the real type of the object.
See also:
C++ FAQ Lite 20.6

Implements LearnModel.

Definition at line 72 of file nnlayer.h.

References NNLayer::NNLayer().

Referenced by FeedForwardNN::add_top().

virtual NNLayer* create (   )  const [inline, virtual]
 

Create a new object using the default constructor.

The code for a derived class Derived is always 

 return new Derived();

Implements LearnModel.

Definition at line 71 of file nnlayer.h.

References NNLayer::NNLayer().

void feed_forward (  const Input &  ,
Output & 
)  const
 

Definition at line 78 of file nnlayer.cpp.

References LearnModel::_n_in, LearnModel::_n_out, lemga::op::inner_product(), LearnModel::n_input(), LearnModel::n_output(), NNLayer::sig_der, NNLayer::sigmoid(), NNLayer::sigmoid_deriv(), and NNLayer::w.

Referenced by NNLayer::operator()().

const WVEC& gradient (   )  const [inline]
 

Definition at line 83 of file nnlayer.h.

References NNLayer::dw.

virtual const id_t& id (   )  const [virtual]
 

Returns:
Class ID string (class name)

Implements Object.

void initialize (   )  [virtual]
 

Reimplemented from LearnModel.

Definition at line 62 of file nnlayer.cpp.

References NNLayer::clear_gradient(), randu, NNLayer::w, NNLayer::w_max, and NNLayer::w_min.

virtual Output operator() (  const Input &  x  )  const [inline, virtual]
 

Implements LearnModel.

Definition at line 88 of file nnlayer.h.

References NNLayer::feed_forward(), and LearnModel::n_output().

bool serialize (  std::ostream &  ,
ver_list & 
)  const [protected, virtual]
 

Reimplemented from LearnModel.

Definition at line 23 of file nnlayer.cpp.

References LearnModel::_n_in, LearnModel::_n_out, SERIALIZE_PARENT, NNLayer::w, NNLayer::w_max, and NNLayer::w_min.

void set_weight (  const WVEC &   ) 
 

Definition at line 53 of file nnlayer.cpp.

References LearnModel::_n_in, LearnModel::_n_out, and NNLayer::w.

void set_weight_range (  REAL  min,
REAL  max
)  [inline]
 

Definition at line 76 of file nnlayer.h.

References NNLayer::w_max, and NNLayer::w_min.

REAL sigmoid (  REAL   )  const [protected, virtual]
 

Definition at line 68 of file nnlayer.cpp.

References quick_tanh().

Referenced by NNLayer::feed_forward().

REAL sigmoid_deriv (  REAL   )  const [protected, virtual]
 

Definition at line 73 of file nnlayer.cpp.

References quick_tanh().

Referenced by NNLayer::feed_forward().

UINT size (   )  const [inline]
 

Definition at line 74 of file nnlayer.h.

References LearnModel::n_output().

virtual void train (   )  [inline, virtual]
 

Train with preset data set and sample weight.

Implements LearnModel.

Definition at line 87 of file nnlayer.h.

References OBJ_FUNC_UNDEFINED.

bool unserialize (  std::istream &  ,
ver_list &  ,
const id_t &  = NIL_ID
)  [protected, virtual]
 

Reimplemented from LearnModel.

Definition at line 35 of file nnlayer.cpp.

References LearnModel::_n_in, LearnModel::_n_out, NNLayer::dw, Object::NIL_ID, NNLayer::sig_der, UNSERIALIZE_PARENT, NNLayer::w, NNLayer::w_max, and NNLayer::w_min.

const WVEC& weight (   )  const [inline]
 

Definition at line 80 of file nnlayer.h.

References NNLayer::w.

Member Data Documentation

WVEC dw [protected]
 

deravatives: w -= lr * dw

Definition at line 63 of file nnlayer.h.

Referenced by NNLayer::back_propagate(), NNLayer::clear_gradient(), NNLayer::gradient(), and NNLayer::unserialize().

DVEC sig_der [mutable, protected]
 

Definition at line 64 of file nnlayer.h.

Referenced by NNLayer::back_propagate(), NNLayer::feed_forward(), and NNLayer::unserialize().

WVEC w [protected]
 

weights and thresholds

Definition at line 62 of file nnlayer.h.

Referenced by NNLayer::back_propagate(), NNLayer::feed_forward(), NNLayer::initialize(), NNLayer::serialize(), NNLayer::set_weight(), NNLayer::unserialize(), and NNLayer::weight().

REAL w_max [protected]
 

Definition at line 61 of file nnlayer.h.

Referenced by NNLayer::initialize(), NNLayer::serialize(), NNLayer::set_weight_range(), and NNLayer::unserialize().

REAL w_min [protected]
 

Definition at line 61 of file nnlayer.h.

Referenced by NNLayer::initialize(), NNLayer::serialize(), NNLayer::set_weight_range(), and NNLayer::unserialize().

The documentation for this class was generated from the following files:
Generated on Wed Nov 8 08:17:07 2006 for LEMGA by  doxygen 1.4.6