Perceptron Class Reference

Perceptron models a type of artificial neural network that consists of only one neuron, invented by Frank Rosenblatt in 1957. More...

#include <perceptron.h>

Inheritance diagram for Perceptron:

[legend]Collaboration diagram for Perceptron:

[legend]List of all members.


Public Types
typedef std::vector< REAL >	WEIGHT
enum	TRAIN_METHOD { PERCEPTRON, ADALINE, POCKET, POCKET_RATCHET, AVE_PERCEPTRON, ROMMA, ROMMA_AGG, SGD_HINGE, SGD_MLSE, RCD, RCD_BIAS, RCD_GRAD, AVE_PERCEPTRON_RAND, ROMMA_RAND, ROMMA_AGG_RAND, COORDINATE_DESCENT, FIXED_RCD, FIXED_RCD_CONJ, FIXED_RCD_BIAS, FIXED_RCD_CONJ_BIAS, RCD_CONJ, RCD_CONJ_BIAS, RCD_GRAD_BATCH, RCD_GRAD_RAND, RCD_GRAD_BATCH_RAND, RCD_MIXED, RCD_GRAD_MIXED, RCD_GRAD_MIXED_INITRAND, RCD_GRAD_MIXED_BATCH, RCD_GRAD_MIXED_BATCH_INITRAND, RAND_COOR_DESCENT = RCD, RAND_COOR_DESCENT_BIAS = RCD_BIAS, RAND_CONJ_DESCENT = RCD_CONJ, RAND_CONJ_DESCENT_BIAS = RCD_CONJ_BIAS, GRADIENT_COOR_DESCENT_ONLINE = RCD_GRAD }
Public Member Functions
	Perceptron (UINT n_in=0)
	Perceptron (const SVM &)
	Perceptron (std::istream &is)
virtual const id_t &	id () const
virtual Perceptron *	create () const
	Create a new object using the default constructor.
virtual Perceptron *	clone () const
	Create a new object by replicating itself.
WEIGHT	weight () const
void	set_weight (const WEIGHT &)
void	start_with_fld (bool b=true)
void	use_resample (bool s=true)
void	set_train_method (TRAIN_METHOD m)
void	set_parameter (REAL lr, REAL mincst, UINT maxrun)
virtual bool	support_weighted_data () const
	Whether the learning model/algorithm supports unequally weighted data.
virtual void	initialize ()
	Initialize the model for training.
WEIGHT	fld () const
virtual REAL	train ()
	Train with preset data set and sample weight.
virtual Output	operator() (const Input &) const
virtual REAL	margin_norm () const
	The normalization term for margins.
virtual REAL	margin_of (const Input &, const Output &) const
	Report the (unnormalized) margin of an example (x, y).
REAL	w_norm () const
Protected Member Functions
virtual bool	serialize (std::ostream &, ver_list &) const
virtual bool	unserialize (std::istream &, ver_list &, const id_t &=empty_id)
virtual void	log_error (UINT, REAL=-1) const
Protected Attributes
WEIGHT	wgt
	wgt.back() is the bias
bool	resample
	reweighting or resampling
TRAIN_METHOD	train_method
REAL	learn_rate
REAL	min_cst
UINT	max_run
bool	with_fld
	start training with FLD?

Detailed Description

Perceptron models a type of artificial neural network that consists of only one neuron, invented by Frank Rosenblatt in 1957.

We use the convention that $w_0$ is the negative threshold, or equivalently, letting $x_0=1$ . When presented with input $x$ , the perceptron outputs

$o = \theta(s),\quad s=\sum_i w_ix_i,$

where $\theta(\cdot)$ is usually the sign function,

The learning algorithm updates the weight according to

$w^{(t+1)} = w^{(t)} + \eta (y-o) x,$

where $y$ is the desired output. If $\theta$ is the sign, the learning rate $\eta$ can be omitted since it just scales $w$ .

Note:: If is replaced by , the sum before the function $\theta$ , the algorithm is then called the ADALINE learning.

Definition at line 35 of file perceptron.h.

Member Typedef Documentation

typedef std::vector<REAL> WEIGHT

Definition at line 37 of file perceptron.h.

Member Enumeration Documentation

enum TRAIN_METHOD

Enumerator:

PERCEPTRON

ADALINE

POCKET

POCKET_RATCHET

AVE_PERCEPTRON

ROMMA

ROMMA_AGG

SGD_HINGE

SGD_MLSE

RCD

RCD_BIAS

RCD_GRAD

AVE_PERCEPTRON_RAND

ROMMA_RAND

ROMMA_AGG_RAND

COORDINATE_DESCENT

FIXED_RCD

FIXED_RCD_CONJ

FIXED_RCD_BIAS

FIXED_RCD_CONJ_BIAS

RCD_CONJ

RCD_CONJ_BIAS

RCD_GRAD_BATCH

RCD_GRAD_RAND

RCD_GRAD_BATCH_RAND

RCD_MIXED

RCD_GRAD_MIXED

RCD_GRAD_MIXED_INITRAND

RCD_GRAD_MIXED_BATCH

RCD_GRAD_MIXED_BATCH_INITRAND

RAND_COOR_DESCENT

RAND_COOR_DESCENT_BIAS

RAND_CONJ_DESCENT

RAND_CONJ_DESCENT_BIAS

GRADIENT_COOR_DESCENT_ONLINE

Definition at line 38 of file perceptron.h.

Constructor & Destructor Documentation

Perceptron ( UINT n_in = 0 ) [explicit]

Definition at line 196 of file perceptron.cpp.
Referenced by Perceptron::clone(), and Perceptron::create().

Perceptron ( const SVM & )

Definition at line 203 of file perceptron.cpp.
References LearnModel::_n_in, SVM::n_support_vectors(), SVM::support_vector(), SVM::support_vector_coef(), and Perceptron::wgt.

Perceptron ( std::istream & is ) [inline, explicit]

Definition at line 92 of file perceptron.h.

Member Function Documentation

virtual Perceptron* clone ( ) const [inline, virtual]

Create a new object by replicating itself.

Returns:
A pointer to the new copy.
The code for a derived class Derived is always
return new Derived(*this);
Though seemingly redundant, it helps to copy an object without knowing the real type of the object.
See also:
C++ FAQ Lite 20.6

Implements LearnModel.
Definition at line 96 of file perceptron.h.
References Perceptron::Perceptron().

virtual Perceptron* create ( ) const [inline, virtual]

Create a new object using the default constructor.
The code for a derived class Derived is always
return new Derived();

Implements LearnModel.
Definition at line 95 of file perceptron.h.
References Perceptron::Perceptron().

Perceptron::WEIGHT fld ( ) const

Definition at line 253 of file perceptron.cpp.
References LearnModel::_n_in, LearnModel::n_samples, LearnModel::ptd, and LearnModel::ptw.
Referenced by Perceptron::train().

virtual const id_t& id ( ) const [virtual]

Returns:
Class ID string (class name)

Implements Object.

void initialize ( ) [virtual]

Initialize the model for training.

Todo:
whether or when to initialize is a big problem. If we allow "continue_on_learning", then sometimes the initialization is not needed

Note:
this was a virtual function, I changed it to be empty

Reimplemented from LearnModel.
Definition at line 246 of file perceptron.cpp.
References LearnModel::_n_in, PERCEPTRON_FIXED_BIAS, randu, and Perceptron::wgt.

void log_error ( UINT ,

REAL = -1

) const [protected, virtual]

Definition at line 695 of file perceptron.cpp.
References LearnModel::logf, and LearnModel::train_c_error().
Referenced by Perceptron::train().

virtual REAL margin_norm ( ) const [inline, virtual]

The normalization term for margins.
The margin concept can be normalized or unnormalized. For example, for a perceptron model, the unnormalized margin would be the wegithed sum of the input features, and the normalized margin would be the distance to the hyperplane, and the normalization term is the norm of the hyperplane weight.
Since the normalization term is usually a constant, it would be more efficient if it is precomputed instead of being calculated every time when a margin is asked for. The best way is to use a cache. Here I use a easier way: let the users decide when to compute the normalization term.
Reimplemented from LearnModel.
Definition at line 118 of file perceptron.h.
References Perceptron::w_norm().

REAL margin_of ( const Input & ,

const Output &

) const [virtual]

Report the (unnormalized) margin of an example (x, y).

Reimplemented from LearnModel.
Definition at line 685 of file perceptron.cpp.
References INFINITESIMAL, INPUT_SUM, and Perceptron::wgt.

Output operator() ( const Input & ) const [virtual]

Implements LearnModel.
Definition at line 679 of file perceptron.cpp.
References INPUT_SUM, LearnModel::n_input(), and Perceptron::wgt.

bool serialize ( std::ostream & ,

ver_list &

) const [protected, virtual]

Reimplemented from LearnModel.
Definition at line 222 of file perceptron.cpp.
References LearnModel::_n_in, SERIALIZE_PARENT, and Perceptron::wgt.

void set_parameter ( REAL lr,

REAL mincst,

UINT maxrun

) [inline]

Parameters:

lr learning rate.

mincst minimal cost (error) need to be achieved during training.

maxrun maximal # of epochs the training should take.

Definition at line 109 of file perceptron.h.
References Perceptron::learn_rate, Perceptron::max_run, and Perceptron::min_cst.

void set_train_method ( TRAIN_METHOD m ) [inline]

Definition at line 103 of file perceptron.h.
References Perceptron::train_method.

void set_weight ( const WEIGHT & )

Definition at line 239 of file perceptron.cpp.
References LearnModel::_n_in, and Perceptron::wgt.

void start_with_fld ( bool b = true ) [inline]

Definition at line 101 of file perceptron.h.
References Perceptron::with_fld.

virtual bool support_weighted_data ( ) const [inline, virtual]

Whether the learning model/algorithm supports unequally weighted data.

Returns:
true if supporting; false otherwise. The default is false, just for safety.

See also:
set_train_data()

Reimplemented from LearnModel.
Definition at line 112 of file perceptron.h.

REAL train ( ) [virtual]

Train with preset data set and sample weight.

Returns:
Probably the training error.

Todo:
Make the return type void

Implements LearnModel.
Definition at line 334 of file perceptron.cpp.
References Perceptron::ADALINE, lemga::dset_extract(), EPSILON, Perceptron::fld(), GET_XYO, Perceptron::learn_rate, Perceptron::log_error(), Perceptron::max_run, Perceptron::PERCEPTRON, PERCEPTRON_FIXED_BIAS, LearnModel::ptd, LearnModel::ptw, RAND_IDX, Perceptron::resample, SAMPWGT, Perceptron::train_method, Perceptron::wgt, and Perceptron::with_fld.

bool unserialize ( std::istream & ,

ver_list & ,

const id_t & = empty_id

) [protected, virtual]

Reimplemented from LearnModel.
Definition at line 229 of file perceptron.cpp.
References LearnModel::_n_in, Object::empty_id, UNSERIALIZE_PARENT, and Perceptron::wgt.

void use_resample ( bool s = true ) [inline]

Definition at line 102 of file perceptron.h.
References Perceptron::resample.

REAL w_norm ( ) const

Definition at line 690 of file perceptron.cpp.
References DOTPROD_NB, and Perceptron::wgt.
Referenced by Perceptron::margin_norm().

WEIGHT weight ( ) const [inline]

Definition at line 98 of file perceptron.h.
References Perceptron::wgt.

Member Data Documentation

REAL learn_rate [protected]

Definition at line 85 of file perceptron.h.
Referenced by Perceptron::set_parameter(), and Perceptron::train().

UINT max_run [protected]

Definition at line 86 of file perceptron.h.
Referenced by Perceptron::set_parameter(), and Perceptron::train().

REAL min_cst [protected]

Definition at line 85 of file perceptron.h.
Referenced by Perceptron::set_parameter().

bool resample [protected]

reweighting or resampling

Definition at line 83 of file perceptron.h.
Referenced by Perceptron::train(), and Perceptron::use_resample().

TRAIN_METHOD train_method [protected]

Definition at line 84 of file perceptron.h.
Referenced by Perceptron::set_train_method(), and Perceptron::train().

WEIGHT wgt [protected]

wgt.back() is the bias

Definition at line 81 of file perceptron.h.
Referenced by Perceptron::initialize(), Perceptron::margin_of(), Perceptron::operator()(), Perceptron::Perceptron(), Perceptron::serialize(), Perceptron::set_weight(), Perceptron::train(), Perceptron::unserialize(), Perceptron::w_norm(), and Perceptron::weight().

bool with_fld [protected]

start training with FLD?

Definition at line 87 of file perceptron.h.
Referenced by Perceptron::start_with_fld(), and Perceptron::train().

The documentation for this class was generated from the following files:

Generated on Mon Jan 9 23:45:09 2006 for LEMGA by

1.4.6

Perceptron Class Reference

Public Types

Public Member Functions