CGBoost Class Reference

CGBoost (Conjugate Gradient Boosting). More...

#include <cgboost.h>

Inheritance diagram for CGBoost:

[legend]Collaboration diagram for CGBoost:


Public Member Functions
	CGBoost (bool cvx=false, const cost::Cost &c=cost::_cost)
	CGBoost (std::istream &is)
virtual const id_t &	id () const
virtual CGBoost *	create () const
	Create a new object using the default constructor.
virtual CGBoost *	clone () const
	Create a new object by replicating itself.
virtual bool	set_aggregation_size (UINT)
	Specify the number of hypotheses used in aggregating.
virtual void	initialize ()
	Initialize the model for training.
virtual REAL	train ()
	Train with preset data set and sample weight.
virtual REAL	train_gd ()
	Training using gradient-descent.
Protected Member Functions
virtual REAL	linear_weight (const DataWgt &, const LearnModel &)
virtual void	linear_smpwgt (DataWgt &)
virtual bool	serialize (std::ostream &, ver_list &) const
virtual bool	unserialize (std::istream &, ver_list &, const id_t &=empty_id)
Protected Attributes
std::vector< REAL >	ncd
	where $g=\nabla C(F)$
std::vector< REAL >	cgd
	where $d=-g+\beta d_{\mathrm{prev}}$
std::vector< REAL >	cur_err
	data only valid within training (remove?)
Friends
struct	_boost_cg

Detailed Description

CGBoost (Conjugate Gradient Boosting).

This class provides two ways to implement the conjugate gradient idea in the Boosting frame.

The first way is to manipulate the sample weight.

The other way is to adjust the projected search direction f. The adjusted direction is also a linear combination of weak learners. We prefer this way (by use_gradient_descent(true)).

Differences between AdaBoost and CGBoost (gradient descent view):

The weights of all hypotheses (CGBoost), instead of only the weight of the newly added hypothesis (AdaBoost), will be updated during one iteration. Thus in order to ``set_aggregation_size()'' correctly, we have to save weights in every iteration.

Todo:: Documentation

Definition at line 36 of file cgboost.h.

Constructor & Destructor Documentation

CGBoost ( bool cvx = false,

const cost::Cost & c = cost::_cost

) [inline, explicit]

Definition at line 46 of file cgboost.h.
Referenced by CGBoost::clone(), and CGBoost::create().

CGBoost ( std::istream & is ) [inline, explicit]

Definition at line 48 of file cgboost.h.

Member Function Documentation

virtual CGBoost* clone ( ) const [inline, virtual]

Create a new object by replicating itself.

Returns:
A pointer to the new copy.
The code for a derived class Derived is always
return new Derived(*this);
Though seemingly redundant, it helps to copy an object without knowing the real type of the object.
See also:
C++ FAQ Lite 20.6

Reimplemented from Boosting.
Definition at line 52 of file cgboost.h.
References CGBoost::CGBoost().

virtual CGBoost* create ( ) const [inline, virtual]

Create a new object using the default constructor.
The code for a derived class Derived is always
return new Derived();

Reimplemented from Boosting.
Definition at line 51 of file cgboost.h.
References CGBoost::CGBoost().

virtual const id_t& id ( ) const [virtual]

Returns:
Class ID string (class name)

Reimplemented from Boosting.

void initialize ( ) [virtual]

Initialize the model for training.

Todo:
whether or when to initialize is a big problem. If we allow "continue_on_learning", then sometimes the initialization is not needed

Note:
this was a virtual function, I changed it to be empty

Reimplemented from Boosting.
Definition at line 21 of file cgboost.cpp.
References Boosting::initialize().

void linear_smpwgt ( DataWgt & sw ) [protected, virtual]

ncd is actually
$-g/y_i = -\frac{w_i}{y_i} c'_F(F(x_i),y_i) = w_i e^{-y_iF(x_i)}$
It can be iteratively computed as
$ncd_i \leftarrow ncd_i \cdot e^{-\alpha y_i f(x_i)}$

Reimplemented from Boosting.
Definition at line 143 of file cgboost.cpp.
References CGBoost::cur_err, EPSILON, Boosting::get_output(), Boosting::lm_wgt, Aggregating::n_in_agg, LearnModel::n_samples, CGBoost::ncd, LearnModel::ptd, and LearnModel::ptw.

REAL linear_weight ( const DataWgt & ,

const LearnModel & l

) [protected, virtual]

Note:
The sample weight (probability) used in training a new hypothesis is no longer same as the one for calculating the step length, which is exactly the gradient. We have to recover the gradient from CGBoost::ncd.

Reimplemented from Boosting.
Definition at line 111 of file cgboost.cpp.
References LearnModel::c_error(), CGBoost::cur_err, LearnModel::get_output(), LearnModel::n_output(), LearnModel::n_samples, CGBoost::ncd, and LearnModel::ptd.

bool serialize ( std::ostream & ,

ver_list &

) const [protected, virtual]

Reimplemented from Boosting.
Definition at line 38 of file cgboost.cpp.
References Boosting::grad_desc_view, SERIALIZE_PARENT, and Aggregating::size().

bool set_aggregation_size ( UINT n ) [virtual]

Specify the number of hypotheses used in aggregating.

Returns:
false if n is larger than size().
Usually all the hypotheses are used in aggregating. However, a smaller number n can also be specified so that only the first n hypotheses are used.
Reimplemented from Aggregating.
Definition at line 26 of file cgboost.cpp.
References Boosting::clear_cache(), Boosting::grad_desc_view, Boosting::lm_wgt, Aggregating::set_aggregation_size(), and Aggregating::size().

REAL train ( ) [virtual]

Train with preset data set and sample weight.

Returns:
Probably the training error.

Todo:
Make the return type void

Reimplemented from Boosting.
Definition at line 83 of file cgboost.cpp.
References CGBoost::cgd, CGBoost::cur_err, Boosting::grad_desc_view, LearnModel::n_output(), LearnModel::n_samples, CGBoost::ncd, LearnModel::ptw, Boosting::train(), and CGBoost::train_gd().

REAL train_gd ( ) [virtual]

Training using gradient-descent.

Reimplemented from Boosting.
Definition at line 99 of file cgboost.cpp.
References Boosting::convex, Boosting::cost(), and lemga::iterative_optimize().
Referenced by CGBoost::train().

bool unserialize ( std::istream & ,

ver_list & ,

const id_t & = empty_id

) [protected, virtual]

Reimplemented from Boosting.
Definition at line 56 of file cgboost.cpp.
References Object::empty_id, Boosting::lm_wgt, Aggregating::size(), UNSERIALIZE_PARENT, and Boosting::use_gradient_descent().

Friends And Related Function Documentation

friend struct _boost_cg [friend]

Definition at line 38 of file cgboost.h.

Member Data Documentation

std::vector<REAL> cgd [protected]

$d/y_i$ where $d=-g+\beta d_{\mathrm{prev}}$

Definition at line 42 of file cgboost.h.
Referenced by CGBoost::train().

std::vector<REAL> cur_err [protected]

data only valid within training (remove?)

Definition at line 61 of file cgboost.h.
Referenced by CGBoost::linear_smpwgt(), CGBoost::linear_weight(), and CGBoost::train().

std::vector<REAL> ncd [protected]

$-g/y_i$ where $g=\nabla C(F)$

Definition at line 42 of file cgboost.h.
Referenced by CGBoost::linear_smpwgt(), CGBoost::linear_weight(), and CGBoost::train().

The documentation for this class was generated from the following files:

Generated on Mon Jan 9 23:44:41 2006 for LEMGA by

1.4.6

CGBoost Class Reference

Public Member Functions

Protected Member Functions

Protected Attributes

Friends

Detailed Description

Constructor & Destructor Documentation

Member Function Documentation

Friends And Related Function Documentation

Member Data Documentation