SHOGUN  v1.1.0
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
List of all members | Public Member Functions | Public Attributes | Protected Attributes
CStreamingVwFile Class Reference

Detailed Description

Class StreamingVwFile to read vector-by-vector from Vowpal Wabbit data files. It reads the example and label into one object of VwExample type.

Definition at line 26 of file StreamingVwFile.h.

Inheritance diagram for CStreamingVwFile:
Inheritance graph
[legend]

Public Member Functions

 CStreamingVwFile ()
 CStreamingVwFile (char *fname, char rw='r')
virtual ~CStreamingVwFile ()
void set_parser_type (E_VW_PARSER_TYPE type=T_VW)
virtual void get_vector (VwExample *&ex, int32_t &len)
virtual void get_vector_and_label (VwExample *&ex, int32_t &len, float64_t &label)
void set_env (CVwEnvironment *env_to_use)
CVwEnvironmentget_env ()
void set_write_to_cache (bool write_cache)
bool get_write_to_cache ()
virtual bool is_seekable ()
virtual const char * get_name () const
- Public Member Functions inherited from CStreamingFile
 CStreamingFile ()
 CStreamingFile (char *fname, char rw='r')
virtual ~CStreamingFile ()
void close ()
virtual void reset_stream ()
virtual void get_vector (bool *&vector, int32_t &len)
virtual void get_vector (uint8_t *&vector, int32_t &len)
virtual void get_vector (char *&vector, int32_t &len)
virtual void get_vector (int32_t *&vector, int32_t &len)
virtual void get_vector (float32_t *&vector, int32_t &len)
virtual void get_vector (float64_t *&vector, int32_t &len)
virtual void get_vector (int16_t *&vector, int32_t &len)
virtual void get_vector (uint16_t *&vector, int32_t &len)
virtual void get_vector (int8_t *&vector, int32_t &len)
virtual void get_vector (uint32_t *&vector, int32_t &len)
virtual void get_vector (int64_t *&vector, int32_t &len)
virtual void get_vector (uint64_t *&vector, int32_t &len)
virtual void get_vector (floatmax_t *&vector, int32_t &len)
virtual void get_vector_and_label (bool *&vector, int32_t &len, float64_t &label)
virtual void get_vector_and_label (uint8_t *&vector, int32_t &len, float64_t &label)
virtual void get_vector_and_label (char *&vector, int32_t &len, float64_t &label)
virtual void get_vector_and_label (int32_t *&vector, int32_t &len, float64_t &label)
virtual void get_vector_and_label (float32_t *&vector, int32_t &len, float64_t &label)
virtual void get_vector_and_label (float64_t *&vector, int32_t &len, float64_t &label)
virtual void get_vector_and_label (int16_t *&vector, int32_t &len, float64_t &label)
virtual void get_vector_and_label (uint16_t *&vector, int32_t &len, float64_t &label)
virtual void get_vector_and_label (int8_t *&vector, int32_t &len, float64_t &label)
virtual void get_vector_and_label (uint32_t *&vector, int32_t &len, float64_t &label)
virtual void get_vector_and_label (int64_t *&vector, int32_t &len, float64_t &label)
virtual void get_vector_and_label (uint64_t *&vector, int32_t &len, float64_t &label)
virtual void get_vector_and_label (floatmax_t *&vector, int32_t &len, float64_t &label)
virtual void get_string (bool *&vector, int32_t &len)
virtual void get_string (uint8_t *&vector, int32_t &len)
virtual void get_string (char *&vector, int32_t &len)
virtual void get_string (int32_t *&vector, int32_t &len)
virtual void get_string (float32_t *&vector, int32_t &len)
virtual void get_string (float64_t *&vector, int32_t &len)
virtual void get_string (int16_t *&vector, int32_t &len)
virtual void get_string (uint16_t *&vector, int32_t &len)
virtual void get_string (int8_t *&vector, int32_t &len)
virtual void get_string (uint32_t *&vector, int32_t &len)
virtual void get_string (int64_t *&vector, int32_t &len)
virtual void get_string (uint64_t *&vector, int32_t &len)
virtual void get_string (floatmax_t *&vector, int32_t &len)
virtual void get_string_and_label (bool *&vector, int32_t &len, float64_t &label)
virtual void get_string_and_label (uint8_t *&vector, int32_t &len, float64_t &label)
virtual void get_string_and_label (char *&vector, int32_t &len, float64_t &label)
virtual void get_string_and_label (int32_t *&vector, int32_t &len, float64_t &label)
virtual void get_string_and_label (float32_t *&vector, int32_t &len, float64_t &label)
virtual void get_string_and_label (float64_t *&vector, int32_t &len, float64_t &label)
virtual void get_string_and_label (int16_t *&vector, int32_t &len, float64_t &label)
virtual void get_string_and_label (uint16_t *&vector, int32_t &len, float64_t &label)
virtual void get_string_and_label (int8_t *&vector, int32_t &len, float64_t &label)
virtual void get_string_and_label (uint32_t *&vector, int32_t &len, float64_t &label)
virtual void get_string_and_label (int64_t *&vector, int32_t &len, float64_t &label)
virtual void get_string_and_label (uint64_t *&vector, int32_t &len, float64_t &label)
virtual void get_string_and_label (floatmax_t *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector (SGSparseVectorEntry< bool > *&vector, int32_t &len)
virtual void get_sparse_vector (SGSparseVectorEntry< uint8_t > *&vector, int32_t &len)
virtual void get_sparse_vector (SGSparseVectorEntry< char > *&vector, int32_t &len)
virtual void get_sparse_vector (SGSparseVectorEntry< int32_t > *&vector, int32_t &len)
virtual void get_sparse_vector (SGSparseVectorEntry< float32_t > *&vector, int32_t &len)
virtual void get_sparse_vector (SGSparseVectorEntry< float64_t > *&vector, int32_t &len)
virtual void get_sparse_vector (SGSparseVectorEntry< int16_t > *&vector, int32_t &len)
virtual void get_sparse_vector (SGSparseVectorEntry< uint16_t > *&vector, int32_t &len)
virtual void get_sparse_vector (SGSparseVectorEntry< int8_t > *&vector, int32_t &len)
virtual void get_sparse_vector (SGSparseVectorEntry< uint32_t > *&vector, int32_t &len)
virtual void get_sparse_vector (SGSparseVectorEntry< int64_t > *&vector, int32_t &len)
virtual void get_sparse_vector (SGSparseVectorEntry< uint64_t > *&vector, int32_t &len)
virtual void get_sparse_vector (SGSparseVectorEntry< floatmax_t > *&vector, int32_t &len)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< bool > *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< uint8_t > *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< char > *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< int32_t > *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< float32_t > *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< float64_t > *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< int16_t > *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< uint16_t > *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< int8_t > *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< uint32_t > *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< int64_t > *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< uint64_t > *&vector, int32_t &len, float64_t &label)
virtual void get_sparse_vector_and_label (SGSparseVectorEntry< floatmax_t > *&vector, int32_t &len, float64_t &label)
- Public Member Functions inherited from CSGObject
 CSGObject ()
 CSGObject (const CSGObject &orig)
virtual ~CSGObject ()
virtual bool is_generic (EPrimitiveType *generic) const
template<class T >
void set_generic ()
void unset_generic ()
virtual void print_serializable (const char *prefix="")
virtual bool save_serializable (CSerializableFile *file, const char *prefix="")
virtual bool load_serializable (CSerializableFile *file, const char *prefix="")
void set_global_io (SGIO *io)
SGIOget_global_io ()
void set_global_parallel (Parallel *parallel)
Parallelget_global_parallel ()
void set_global_version (Version *version)
Versionget_global_version ()
SGVector< char * > get_modelsel_names ()
char * get_modsel_param_descr (const char *param_name)
index_t get_modsel_param_index (const char *param_name)

Public Attributes

parse_func parse_example
 The function which will be called for parsing.

Protected Attributes

CVwParserparser
 Parser for vw format.
E_VW_PARSER_TYPE parser_type
 Parser type.
CVwEnvironmentenv
 Environment used for vw - used by parser.
bool write_to_cache
 Write data to a binary cache file.
- Protected Attributes inherited from CStreamingFile
CIOBufferbuf
 Buffer to hold stuff in memory.
char task
 Task.
char * filename
 Name of the handled file.

Additional Inherited Members

- Protected Member Functions inherited from CSGObject
virtual void load_serializable_pre () throw (ShogunException)
virtual void load_serializable_post () throw (ShogunException)
virtual void save_serializable_pre () throw (ShogunException)
virtual void save_serializable_post () throw (ShogunException)

Constructor & Destructor Documentation

Default constructor

Definition at line 15 of file StreamingVwFile.cpp.

CStreamingVwFile ( char *  fname,
char  rw = 'r' 
)

Constructor taking file name argument

Parameters
fnamefile name
rwread/write mode

Definition at line 21 of file StreamingVwFile.cpp.

~CStreamingVwFile ( )
virtual

Destructor

Definition at line 27 of file StreamingVwFile.cpp.

Member Function Documentation

CVwEnvironment* get_env ( )

Return the environment

Returns
environment as CVwEnvironment*

Definition at line 94 of file StreamingVwFile.h.

virtual const char* get_name ( ) const
virtual
Returns
object name

Reimplemented from CStreamingFile.

Definition at line 121 of file StreamingVwFile.h.

void get_vector ( VwExample *&  ex,
int32_t &  len 
)
virtual

Returns the parsed example.

The example contains the label if available, and also contains length of the feature vector. These parameters are redundant.

Parameters
exexamples as VwExample*, set by reference
lenlength of vector, untouched

Reimplemented from CStreamingFile.

Definition at line 54 of file StreamingVwFile.cpp.

void get_vector_and_label ( VwExample *&  ex,
int32_t &  len,
float64_t label 
)
virtual

Returns the parsed example.

TODO: Make this fail if examples are found to be unlabelled.

Parameters
exexample as VwExample*, set by reference
lenlength of vector, untouched
labellabel, untouched

Reimplemented from CStreamingFile.

Definition at line 61 of file StreamingVwFile.cpp.

bool get_write_to_cache ( )

Get whether cache will be written

Returns
whether to write to cache

Definition at line 116 of file StreamingVwFile.h.

virtual bool is_seekable ( )
virtual

Whether the stream is seekable/resettable

Returns
false by default, unless overloaded

Reimplemented from CStreamingFile.

Definition at line 118 of file StreamingVwFile.h.

void set_env ( CVwEnvironment env_to_use)

Set environment for vw

Parameters
env_to_useCVwEnvironment* environment

Definition at line 84 of file StreamingVwFile.h.

void set_parser_type ( E_VW_PARSER_TYPE  type = T_VW)

Set the type of parser, i.e., T_VW, T_SVMLIGHT or T_DENSE.

Parameters
typeparser type as enum

Definition at line 33 of file StreamingVwFile.cpp.

void set_write_to_cache ( bool  write_cache)

Set whether cache will be written

Parameters
write_cachewhether to write to cache

Definition at line 105 of file StreamingVwFile.h.

Member Data Documentation

CVwEnvironment* env
protected

Environment used for vw - used by parser.

Definition at line 144 of file StreamingVwFile.h.

parse_func parse_example

The function which will be called for parsing.

Definition at line 128 of file StreamingVwFile.h.

CVwParser* parser
protected

Parser for vw format.

Definition at line 138 of file StreamingVwFile.h.

E_VW_PARSER_TYPE parser_type
protected

Parser type.

Definition at line 141 of file StreamingVwFile.h.

bool write_to_cache
protected

Write data to a binary cache file.

Definition at line 147 of file StreamingVwFile.h.


The documentation for this class was generated from the following files:

SHOGUN Machine Learning Toolbox - Documentation