Romain Francois, Professional R Enthusiast

I've pushed a new version of the parser package to CRAN.

This is the first release that depends on Rcpp, which allowed me to reduce the code size and increase its maintainability.

This also features a faster version of nlines, a function that retrieves the number of lines of a text file.

I've been playing with images recently, probably inspired from my trip in San Francisco. There was an exhibit at the De Young museum of fine arts with pieces borrowed from the Musée d'Orsay. I did not go to the exhibit because it is easy enough for me to just go to Paris and the Musée d'Orsay, but I guess this somewhat inspired me, along with the golden gate bridge, to do some R based impressionnism

The starting point is this picture of the golden gate bridge

The png package makes it straightforward to import png pictures into R (There are other ways as well).

Then, I generate randomly spaced circles so that they don't overlap, and fill each circle with the average color (on the RGB space) of all te pixels that are inside the circle

Then I do this many times, with translucent circles, and after some iterations ,the golden gate bridge starts to reveal itself

The code for this is included below

Here are other examples

Dirk uploaded Rcpp 0.8.4 to CRAN yesterday. This release quickly follows the release of Rcpp 0.8.3, because there was some building problems (particularly on the ppc arch on OSX).

Rcpp sugar

Already available in Rcpp 0.8.3, the new sugar feature was extended in 0.8.4 to cover more functions, and we have now started to adapt sugar for matrices with functions such as outer, row, diag, etc ...

Here is an example of using the sugar version of outer

NumericVector xx(x) ;
NumericVector yy(y);
NumericMatrix m = outer( xx, yy, std::plus<double>() ) ;
return m ;

This mimics the R code

> outer( x, y, "+" )

Here is the relevant extract of the NEWS file:

0.8.4   2010-07-09

o   new sugar vector functions: rep, rep_len, rep_each, rev, head, tail, diag
	
o	sugar has been extended to matrices: The Matrix class now extends the Matrix_Base template that implements CRTP. Currently sugar functions for matrices are: outer, col, row, lower_tri, upper_tri, diag

o   The unit tests have been reorganised into fewer files with one call each to cxxfunction() (covering multiple tests) resulting in a significant speedup

o	The Date class now uses the same mktime() replacement that R uses (based on original code from the timezone library by Arthur Olson) permitting wide dates ranges on all operating systems

o   The FastLM/example has been updated, a new benchmark based on the historical Longley data set has been added

o   RcppStringVector now uses std::vector<std::string> internally

o    setting the .Data slot of S4 objects did not work properly

We released Rcpp 0.8.0 almost a month ago. It finalized our efforts in designing a better, faster and more natural API than any version of Rcpp ever before. The journey from Rcpp 0.7.0 to Rcpp 0.8.0 has mainly been a coding and testing effort for designing the API.

And now for something completely different

We have now started (with release 0.8.1 of Rcpp) a new development cycle towards the 0.9.0 version with two major goals in mind

• We want to improve documentation. To that end Rcpp 0.8.1 includes 4 new vignettes. more on that later.
• We want to cross the boundaries between R and C++. Rcpp 0.8.1 introduces Rcpp modules. Modules allows the programmer to expose C++ classes and functions at the R level, with great ease.

new vignettes

Rcpp-FAQ :Frequently Asked Questions about Rcpp collects some of the frequently asked questions from the mailing list and from private exchanges with many people.

Rcpp-extending: Extending Rcpp shows how to extend Rcpp converters Rcpp::wrap and Rcpp::as to user defined types (C++ classes defined in someone else's package and third party types (C++ classes defined in some third party library used by a package. The document is based on our experience developping the RcppArmadillo package

Rcpp-package : Writing a package that uses Rcpp highlights the steps involved in making a package that uses Rcpp. The document is based on the Rcpp.package.skeleton function

finally, Rcpp-modules : Exposing C++ functions and classes with Rcpp modules documents the current feature set of Rcpp modules

Rcpp modules

Rcpp modules are inspired from the Boost.Python C++ library. Rcpp modules let you expose C++ classes and functions to the R level with minimal involvment from the programmer

The feature is best described by an example (more examples on the vignette). Say we want to expose this simple class:

This would typically involve external pointers. With Rcpp modules, we can simply declare what we want to expose about this class, and Rcpp takes care of the how to expose it:

The R side consists of grabbing a reference to the module, and just use the World class

The Rcpp-modules vignette gives more details about modules, including how to use them in packages

More details about 0.8.1 release

Here is the complete extract from our NEWS file about this release

I've released version 0.2-0 of highlight to CRAN

This version brings some more additions to the sweave driver that uses highlight to produce nice looking vignettes with color coded R chunks

The driver gains new arguments boxes, bg and border to control the appearance of the code chunks. When boxes is set to TRUE, the R code chunks are surrounded in boxes, and the arguments bg and border control the background color and the color of the box

Also, when the other highlight is available, the driver will also color code example code in any language that highlight supports. To use this, just surorund the code with <<lang=foo>> for the language foo. For example:

 <<lang=cpp>>=
int main(){
return 0 ;
}
@

will output the content of the code chunk as highlighted c++. The Rcpp-modules vignette in the next version of Rcpp uses both these new features. (see the vignette source in r-forge. The vignette is rendered into latex using :

require(highlight)
driver <- HighlightWeaveLatex(boxes = TRUE)
Sweave( 'Rcpp-modules.Rnw', driver = driver )

Summary

Version 0.8.0 of the Rcpp package was released to CRAN today. This release marks another milestone in the ongoing redesign of the package, and underlying C++ library.

Overview

Rcpp is an R package and C++ library that facilitates integration of C++ code in R packages.

The package features a set of C++ classes (Rcpp::IntegerVector, Rcpp::Function, Rcpp::Environment, ...) that makes it easier to manipulate R objects of matching types (integer vectors, functions, environments, etc ...).

Rcpp takes advantage of C++ language features such as the explicit constructor/destructor lifecycle of objects to manage garbage collection automatically and transparently. We believe this is a major improvement over PROTECT/UNPROTECT. When an Rcpp object is created, it protects the underlying SEXP so that the garbage collector does not attempt to reclaim the memory. This protection is withdrawn when the object goes out of scope. Moreover, users generally do not need to manage memory directly (via calls to new / delete or malloc / free) as this is done by the Rcpp classes or the corresponding STL containers.

API

Rcpp provides two APIs: an older set of classes we refer to the classic API (see below for the section 'Backwards Compatibility) as well as second and newer set of classes.

Classes of the new Rcpp API belong to the Rcpp namespace. Each class is associated to a given SEXP type and exposes an interface that allows manipulation of the object that may feel more natural than the usual use of macros and functions provided by the R API.

Some SEXP types do not have dedicated Rcpp classes : NILSXP, DOTSXP, ANYSXP, BCODESXP and CHARSXP.

Still missing are a few convenience classes such as Rcpp::Date or Rcpp::Datetime which would map useful and frequently used R data types, but which do not have an underlying SEXP type.

Data Interchange

Data interchange between R and C++ is managed by extensible and powerful yet simple mechanisms.

Conversion of a C++ object is managed by the template function Rcpp::wrap. This function currently manages :

• primitive types : int, double, bool, float, Rbyte, ...
• std::string, const char*
• STL containers such as std::vector<T> and STL maps such as std::mapr< std::string, Tr> provided that the template type T is wrappable
• any class that can be implicitely converted to SEXP, through operator SEXP()

Conversion of an R object to a C++ object is managed by the Rcpp::as<T> template which can handle:

• primitive types
• std::string, const char*
• STL containers such as std::vector<T>

Rcpp::wrap and Rcpp::as are often used implicitely. For example, when assigning objects to an environment:

  // grab the global environment
  Rcpp::Environment global = Rcpp::Environment::global_env() ;
  std::deque z( 3 ); z[0] = false; z[1] = true; z[3] = false ;

  global["x"] = 2 ;                    // implicit call of wrap
  global["y"] = "foo";                 // implicit call of wrap
  global["z"] = z ;                    // impl. call of wrap>

  int x = global["x"] ;                // implicit call of as
  std::string y = global["y"]          // implicit call of as
  std::vector z1 = global["z"] ; // impl. call of as>

Rcpp contains several examples that illustrate wrap and as. The mechanism was designed to be extensible. We have developped separate packages to illustrate how to extend Rcpp conversion mechanisms to third party types.

• RcppArmadillo : conversion of types from the Armadillo C++ library.
• RcppGSL : conversion of types from the GNU Scientific Library.

Rcpp is also used for data interchange by the RInside package which provides and easy way of embedding an R instance inside of C++ programs.

inline use

Rcpp depends on the inline package by Oleg Sklyar et al. Rcpp then uses the 'cfunction' provided by inline (with argument Rcpp=TRUE) to compile, link and load C++ function from the R session.

As of version 0.8.0 of Rcpp, we also define an R function cppfunction that acts as a facade function to the inline::cfuntion, with specialization for C++ use.

This allows quick prototyping of compiled code. All our unit tests are based on cppfunction and can serve as examples of how to use the mechanism. For example this function (from the runit.GenericVector.R unit test file) defines from R a C++ (simplified) version of lapply:

  ## create a compiled function cpp_lapply using cppfunction 
  cpp_lapply <- cppfunction(signature(x = "list", g = "function" ), 
  		'Function fun(g) ;
		 List input(x) ;
		 List output( input.size() ) ;
		 std::transform( input.begin(), input.end(), output.begin(), fun ) ;
		 output.names() = input.names() ;
		 return output ;
	    ')
  ## call cpp_lapply on the iris data with the R function summary
  cpp_lapply( iris, summary )	

Using Rcpp in other packages

Rcpp is designed so that its classes are used from other packages. Using Rcpp requires :

• using the header files provided by Rcpp. This is typically done by adding this line in the package DESRIPTION file:

  	LinkingTo: Rcpp
  

  and add the following line in the package code:

  	#include <Rcpp.h>
  

• linking against the Rcpp dynamic or static library, which is achieved by adding this line to the src/Makevars of the package:

  	PKG_LIBS = $(shell $(R_HOME)/bin/Rscript -e "Rcpp:::LdFlags()" )
  

  and this line to the src/Makevars.win file:

  	PKG_LIBS = $(shell Rscript.exe -e "Rcpp:::LdFlags()")
  

Rcpp contains a function Rcpp.package.skeleton, modelled after package.skeleton from the utils package in base r, that creates a skeleton of a package using Rcpp, including example code.

C++ exceptions

C++ exceptions are R contexts are both based on non local jumps (at least on the implementation of exceptions in gcc), so care must be ensure that one system does not void assumptions of the other. It is therefore very strongly recommended that each function using C++ catches C++ exceptions. Rcpp offers the function forward_exception_to_r to facilitate forwarding the exception to the "R side" as an R condition. For example :

  SEXP foo( ) {
    try {
      // user code here
    } catch( std::exception& __ex__){
      forward_exception_to_r( __ex__ ) ;
    }
    // return something here
  }

Alternatively, functions can enclose the user code with the macros BEGIN_RCPP and END_RCPP, which provides for a more compact way of programming. The function above could be written as follows using the macros:

  SEXP foo( ) {
    BEGIN_RCPP
    // user code here
    END_RCPP
    // return something here
  }

The use of BEGIN_RCPP and END_RCPP is recommended to anticipate future changes of Rcpp. We might for example decide to install dedicated handlers for specific exceptions later.

Experimental code generation macros

Rcpp contains several macros that can generate repetitive 'boiler plate' code:

  RCPP_FUNCTION_0, ..., RCPP_FUNCTION_65
  RCPP_FUNCTION_VOID_0, ..., RCPP_FUNCTION_VOID_65
  RCPP_XP_METHOD_0, ..., RCPP_XP_METHOD_65
  RCPP_XP_METHOD_CAST_0, ..., RCPP_XP_METHOD_CAST_65
  RCPP_XP_METHOD_VOID_0, ..., RCPP_XP_METHOD_VOID_65

For example:

  RCPP_FUNCTION_2( int, foobar, int x, int y){
     return x + y ;
  }

This will create a .Call compatible function "foobar" that calls a c++ function for which we provide the argument list (int x, int y) and the return type (int). The macro also encloses the call in BEGIN_RCPP/END_RCPP so that exceptions are properly forwarded to R.

Examples of the other macros are given in the NEWS file.

This feature is still experimental, but is being used in packages highlight and RProtoBuf

Quality Assurance

Rcpp uses the RUnit package by Matthias Burger et al and the aforementioned inline package by Oleg Sklyar et al to provide unit testing. Rcpp currently has over 500 unit tests (called from more than 230 unit test functions) with very good coverage of the critical parts of the package and library.

Source code for unit test functions are stored in the unitTests directory of the installed package and the results are collected in the "Rcpp-unitTests" vignette.

The unit tests can be both during the standard R package build and testing process, and also when the package is installed. The latter use is helpful to ensure that no system components have changed in a way that affect the Rcpp package since it has been installed. To run the tests, execute

   Rcpp:::test()

where an output directory can be provided as an optional first argument.

Backwards Compatibility

We believe the new API is now more complete and useful than the previous set of classes, which we refer to as the "classic Rcpp API". We would therefore recommend to package authors using 'classic' Rcpp to move to the new API. However, the classic API is still maintained and will continue to be maintained to ensure backwards compatibility for code that uses it.

Packages uses the 'Classic API' can use features of the new API selectively and in incremental steps. This provides for a non-disruptive upgrade path.

Documentation

The package contains a vignette which provides a short and succinct introduction to the Rcpp package along with several motivating examples. Also provided is a vignette containing the regression test summary from the time the package was built.

Links

• Rcpp main page: http://dirk.eddelbuettel.com/code/rcpp.html
• R-forge project page: http://r-forge.r-project.org/projects/rcpp/
• Dirk's blog section about Rcpp: http://dirk.eddelbuettel.com/blog/code/rcpp/
• Romain's blog section about Rcpp: http://romainfrancois.blog.free.fr/index.php?category/R-package/Rcpp

Support

Questions about Rcpp should be directed to the Rcpp-devel mailing list

 -- Dirk Eddelbuettel and Romain Francois
    Chicago, IL, USA, and Montpellier, France
	May 2010

The next version of R includes support for raster images in standard and grid graphics.

The RImageJ package uses ImageJ through rJava to read and manipulate images from various formats

Paul Murrell closed the gap and contributed code that allows using images from the RImageJ package as raster objects.

makes the graph :

This feature depends on R >= 2.11.0, so will only get available when this version becomes current, in the meantime, you can get the package from its rforge project page

I've pushed the version 0.1-5 of highlight to CRAN, it should be available in a couple of days.

This version fixes highlighting of code when one wants to display the prompt and the continue prompt. For example, this code :

rnorm(10, 
	mean = 5)


runif(5)

gets highlighted like this:

using this code:

> highlight( "/tmp/test.R", renderer=renderer_html(document=T), showPrompts = TRUE, output = "test.html" )

Under the hood, highlight now depends on Rcpp and uses some of the C++ classes of the new Rcpp API. See the get_highlighted_text function in the code.

Dirk released Rcpp 0.7.5 yesterday

The main thing is the smarter wrap function that now uses techniques of type traits and template meta-programming to have a compile time guess at whether an object is wrappable, and how to do it. Currently wrappable types are :

• primitive types : int, double, Rbyte, Rcomplex
• std::string
• STL containers such as std::vector<T> as long as T is wrappable. This is not strictly tied to the STL, actually any type that has a nested type called iterator and member functions begin() and end() will do
• STL maps keyed by strings such as std::map<std::string,T> as long as T is wrappable
• any class that can be implicitely converted to SEXP
• any class for which the wrap template is partly or fully specialized. (The next version of RInside has an example of that)

Here comes an example (from our unit tests) :

        funx <- cfunction(signature(), 
        '
        std::map< std::string,std::vector<int> > m ;
        std::vector<int> b ; b.push_back(1) ; b.push_back(2) ; m["b"] = b ;
        std::vector<int> a ; a.push_back(1) ; a.push_back(2) ; a.push_back(2) ; m["a"] = a ;
        std::vector<int> c ; c.push_back(1) ; c.push_back(2) ; c.push_back(2) ; c.push_back(2) ; m["c"] = c ;
        return wrap(m) ;
        ', 
        Rcpp=TRUE, verbose=FALSE, includes = "using namespace Rcpp;" )

R> funx()
$a
[1] 1 2 2

$b
[1] 1 2

$c
[1] 1 2 2 2

Apart from that, other things have changed, here is the relevant section of the NEWS for this release

    o 	wrap has been much improved. wrappable types now are :
    	- primitive types : int, double, Rbyte, Rcomplex, float, bool
    	- std::string
    	- STL containers which have iterators over wrappable types:
    	  (e.g. std::vector, std::deque, std::list, etc ...). 
    	- STL maps keyed by std::string, e.g std::map
    	- classes that have implicit conversion to SEXP
    	- classes for which the wrap template if fully or partly specialized
    	This allows composition, so for example this class is wrappable: 
    	std::vector< std::map > (if T is wrappable)
    	
    o 	The range based version of wrap is now exposed at the Rcpp::
    	level with the following interface : 
    	Rcpp::wrap( InputIterator first, InputIterator last )
    	This is dispatched internally to the most appropriate implementation
    	using traits

    o	a new namespace Rcpp::traits has been added to host the various
    	type traits used by wrap

    o 	The doxygen documentation now shows the examples

    o 	A new file inst/THANKS acknowledges the kind help we got from others

    o	The RcppSexp has been removed from the library.
    
    o 	The methods RObject::asFoo are deprecated and will be removed
    	in the next version. The alternative is to use as.

    o	The method RObject::slot can now be used to get or set the 
    	associated slot. This is one more example of the proxy pattern
    	
    o	Rcpp::VectorBase gains a names() method that allows getting/setting
    	the names of a vector. This is yet another example of the 
    	proxy pattern.
    	
    o	Rcpp::DottedPair gains templated operator<< and operator>> that 
    	allow wrap and push_back or wrap and push_front of an object
    	
    o	Rcpp::DottedPair, Rcpp::Language, Rcpp::Pairlist are less
    	dependent on C++0x features. They gain constructors with up
    	to 5 templated arguments. 5 was choosed arbitrarily and might 
    	be updated upon request.
    	
    o	function calls by the Rcpp::Function class is less dependent
    	on C++0x. It is now possible to call a function with up to 
    	5 templated arguments (candidate for implicit wrap)
    	
    o	added support for 64-bit Windows (thanks to Brian Ripley and Uwe Ligges)

Rcpp 0.7.2 is out, checkout Dirk's blog for details

selected highlights from this new version:

character vectors

if one wants to mimic this R code in C

> x <- c( "foo", "bar" )

one ends up with this :

SEXP x = PROTECT( allocVector( STRSXP, 2) ) ;
SET_STRING_ELT( x, 0, mkChar( "foo" ) ) ;
SET_STRING_ELT( x, 1, mkChar( "bar" ) ) ;
UNPROTECT(1) ;
return x ;

Rcpp lets you express the same like this :

CharacterVector x(2) ;
x[0] = "foo" ; 
x[1] = "bar" ;

or like this if you have GCC 4.4 :

CharacterVector x = { "foo", "bar" } ;

environments, functions, ...

Now, we try to mimic this R code in C :

rnorm( 10L, sd = 100 )

You can do one of these two ways in Rcpp :

Environment stats("package:stats") ;
Function rnorm = stats.get( "rnorm" ) ;
return rnorm( 10, Named("sd", 100 ) ) ;

or :

Language call( "rnorm", 10, Named("sd", 100 ) ) ;
return eval( call, R_GlobalEnv ) ;

and it will get better with the next release, where you will be able to just call call.eval() and stats["rnorm"].

Using the regular R API, you'd write these liks this :

SEXP stats = PROTECT( R_FindNamespace( mkString("stats") ) ) ;
SEXP rnorm = PROTECT( findVarInFrame( stats, install("rnorm") ) ) ;
SEXP call  = PROTECT( LCONS( rnorm, CONS(ScalarInteger(10), CONS(ScalarReal(100.0), R_NilValue)))) ;
SET_TAG( CDDR(call), install("sd") ) ;
SEXP res = PROTECT( eval( call, R_GlobalEnv ) );
UNPROTECT(4) ;
return res ;

or :

SEXP call  = PROTECT( LCONS( install("rnorm"), CONS(ScalarInteger(10), CONS(ScalarReal(100.0), R_NilValue)))) ;
SET_TAG( CDDR(call), install("sd") ) ;
SEXP res = PROTECT( eval( call, R_GlobalEnv ) );
UNPROTECT(2) ;
return res ;