CGBoost Class Reference

CGBoost (Conjugate Gradient Boosting). More...

#include <cgboost.h>

Inheritance diagram for CGBoost:

[legend]Collaboration diagram for CGBoost:


Public Member Functions
	CGBoost (bool cvx=false, const cost::Cost &c=cost::_cost)
	CGBoost (const Boosting &s)
	CGBoost (std::istream &is)
virtual const id_t &	id () const
virtual CGBoost *	create () const
	Create a new object using the default constructor.
virtual CGBoost *	clone () const
	Create a new object by replicating itself.
virtual bool	set_aggregation_size (UINT)
	Specify the number of hypotheses used in aggregating.
virtual void	train ()
	Train with preset data set and sample weight.
virtual void	reset ()
Protected Member Functions
virtual void	train_gd ()
	Training using gradient-descent.
virtual REAL	linear_weight (const DataWgt &, const LearnModel &)
virtual void	linear_smpwgt (DataWgt &)
virtual bool	serialize (std::ostream &, ver_list &) const
virtual bool	unserialize (std::istream &, ver_list &, const id_t &=NIL_ID)
Protected Attributes
std::vector< REAL >	ncd
	where $g=\nabla C(F)$
std::vector< REAL >	cgd
	where $d=-g+\beta d_{\mathrm{prev}}$
std::vector< REAL >	cur_err
	data only valid within training (remove?)
Friends
struct	_boost_cg

Detailed Description

CGBoost (Conjugate Gradient Boosting).

This class provides two ways to implement the conjugate gradient idea in the Boosting frame.

The first way is to manipulate the sample weight.

The other way is to adjust the projected search direction f. The adjusted direction is also a linear combination of weak learners. We prefer this way (by use_gradient_descent(true)).

Differences between AdaBoost and CGBoost (gradient descent view):

The weights of all hypotheses (CGBoost), instead of only the weight of the newly added hypothesis (AdaBoost), will be updated during one iteration. Thus in order to ``set_aggregation_size()'' correctly, we have to save weights in every iteration.

Todo:: Documentation

Definition at line 36 of file cgboost.h.

Constructor & Destructor Documentation

CGBoost ( bool cvx = false,

const cost::Cost & c = cost::_cost

) [inline, explicit]

Definition at line 46 of file cgboost.h.
Referenced by CGBoost::clone(), and CGBoost::create().

CGBoost ( const Boosting & s ) [inline]

Definition at line 48 of file cgboost.h.
References Boosting::lm_wgt.

CGBoost ( std::istream & is ) [inline, explicit]

Definition at line 53 of file cgboost.h.

Member Function Documentation

virtual CGBoost* clone ( ) const [inline, virtual]

Create a new object by replicating itself.

Returns:
A pointer to the new copy.
The code for a derived class Derived is always
return new Derived(*this);
Though seemingly redundant, it helps to copy an object without knowing the real type of the object.
See also:
C++ FAQ Lite 20.6

Reimplemented from Boosting.
Definition at line 57 of file cgboost.h.
References CGBoost::CGBoost().

virtual CGBoost* create ( ) const [inline, virtual]

Create a new object using the default constructor.
The code for a derived class Derived is always
return new Derived();

Reimplemented from Boosting.
Definition at line 56 of file cgboost.h.
References CGBoost::CGBoost().

virtual const id_t& id ( ) const [virtual]

Returns:
Class ID string (class name)

Reimplemented from Boosting.

void linear_smpwgt ( DataWgt & sw ) [protected, virtual]

ncd is actually
$-g/y_i = -\frac{w_i}{y_i} c'_F(F(x_i),y_i) = w_i e^{-y_iF(x_i)}$
It can be iteratively computed as
$ncd_i \leftarrow ncd_i \cdot e^{-\alpha y_i f(x_i)}$

Reimplemented from Boosting.
Definition at line 141 of file cgboost.cpp.
References CGBoost::cur_err, EPSILON, Boosting::get_output(), Boosting::lm_wgt, Aggregating::n_in_agg, LearnModel::n_samples, CGBoost::ncd, LearnModel::ptd, and LearnModel::ptw.

REAL linear_weight ( const DataWgt & ,

const LearnModel & l

) [protected, virtual]

Note:
The sample weight (probability) used in training a new hypothesis is no longer same as the one for calculating the step length, which is exactly the gradient. We have to recover the gradient from CGBoost::ncd.

Reimplemented from Boosting.
Definition at line 108 of file cgboost.cpp.
References LearnModel::c_error(), CGBoost::cur_err, LearnModel::exact_dimensions(), LearnModel::get_output(), LearnModel::n_samples, CGBoost::ncd, LearnModel::ptd, and LearnModel::train_data().

void reset ( ) [virtual]

Delete learning models stored in lm. This is only used in operator= and load().
Note:
lm_base is not deleted since load() will need it

Todo:
make it public under the name clear()? Or remove it

Reimplemented from Boosting.
Definition at line 21 of file cgboost.cpp.
References Boosting::reset().

bool serialize ( std::ostream & ,

ver_list &

) const [protected, virtual]

Reimplemented from Boosting.
Definition at line 38 of file cgboost.cpp.
References Boosting::grad_desc_view, SERIALIZE_PARENT, and Aggregating::size().

bool set_aggregation_size ( UINT n ) [virtual]

Specify the number of hypotheses used in aggregating.

Returns:
false if n is larger than size().
Usually all the hypotheses are used in aggregating. However, a smaller number n can also be specified so that only the first n hypotheses are used.
Reimplemented from Aggregating.
Definition at line 26 of file cgboost.cpp.
References Boosting::clear_cache(), Boosting::grad_desc_view, Boosting::lm_wgt, Aggregating::set_aggregation_size(), and Aggregating::size().

void train ( ) [virtual]

Train with preset data set and sample weight.

Reimplemented from Boosting.
Definition at line 83 of file cgboost.cpp.
References CGBoost::cgd, CGBoost::cur_err, Boosting::grad_desc_view, LearnModel::n_samples, CGBoost::ncd, LearnModel::ptw, and Boosting::train().

void train_gd ( ) [protected, virtual]

Training using gradient-descent.

Reimplemented from Boosting.
Definition at line 97 of file cgboost.cpp.
References Boosting::convex, and lemga::iterative_optimize().

bool unserialize ( std::istream & ,

ver_list & ,

const id_t & = NIL_ID

) [protected, virtual]

Reimplemented from Boosting.
Definition at line 56 of file cgboost.cpp.
References Boosting::lm_wgt, Object::NIL_ID, Aggregating::size(), UNSERIALIZE_PARENT, and Boosting::use_gradient_descent().

Friends And Related Function Documentation

friend struct _boost_cg [friend]

Definition at line 38 of file cgboost.h.

Member Data Documentation

std::vector<REAL> cgd [protected]

$d/y_i$ where $d=-g+\beta d_{\mathrm{prev}}$

Definition at line 42 of file cgboost.h.
Referenced by CGBoost::train().

std::vector<REAL> cur_err [protected]

data only valid within training (remove?)

Definition at line 65 of file cgboost.h.
Referenced by CGBoost::linear_smpwgt(), CGBoost::linear_weight(), and CGBoost::train().

std::vector<REAL> ncd [protected]

$-g/y_i$ where $g=\nabla C(F)$

Definition at line 42 of file cgboost.h.
Referenced by CGBoost::linear_smpwgt(), CGBoost::linear_weight(), and CGBoost::train().

The documentation for this class was generated from the following files:

Generated on Wed Nov 8 08:16:48 2006 for LEMGA by

1.4.6

CGBoost Class Reference

Public Member Functions

Protected Member Functions

Protected Attributes

Friends

Detailed Description

Constructor & Destructor Documentation

Member Function Documentation

Friends And Related Function Documentation

Member Data Documentation