Romain Francois, Professional R Enthusiast

To content | To menu | To search

Friday, August 28 2009

Combine R CMD build and junit

This is a post in the series Mixing R CMD build and ant. Previous posts have shown how to compile the java code that lives in the src directory of a package and how to document this code using javadoc.

This post tackles the problem of unit testing of the java functionality that is shipped as part of an R package. Java has several unit test platforms, we will use junit here, but similar things could be done with other systems such as testng, ...

The helloJavaWorld package now looks like this :

.
|-- DESCRIPTION
|-- NAMESPACE
|-- R
|   |-- helloJavaWorld.R
|   `-- onLoad.R
|-- inst
|   |-- doc
|   |   |-- helloJavaWorld.Rnw
|   |   |-- helloJavaWorld.pdf
|   |   `-- helloJavaWorld.tex
|   `-- java
|       |-- hellojavaworld-tests.jar
|       `-- hellojavaworld.jar
|-- man
|   `-- helloJavaWorld.Rd
`-- src
    |-- Makevars
    |-- build.xml
    |-- junit
    |   `-- HelloJavaWorld_Test.java
    |-- lib
    |   `-- junit-4.7.jar
    `-- src
        `-- HelloJavaWorld.java

9 directories, 15 files

We have added the src/lib directory that contains the junit library and the HelloJavaWorld_Test.java that contain a simple class with a unit test

And the ant build file has been changed in order to

  • build the junit test cases, see the build-testcases target
  • run the unit tests, see the test target
  • create nice html reports, see the report target

The package can be downloaded here

Coming next, handling of dependencies between java code that lives in different R packages

Sunday, August 9 2009

Completion for java objects

As indicated in this thread, completion after the dollar operator can be customized by defining a custom names method. Here I am showing how to take advantage of this to display fields and methods of java references (jobjRef objects from the rJava package)

Here it is in action (I hit tab twice after the dollar sign)

Wednesday, August 5 2009

Code Snippet : List of CRAN packages

This is a really simple code snippet that shows how to get the list of CRAN packages and their titles from the html page html page (toulouse mirror in this example).

...

Note that R has the available.packages function, but it does not give the titles of the packages

Tuesday, August 4 2009

R parser package on CRAN

The parser package has been released to CRAN, the package mainly defines a function parser that is similar to the usual R function parse, with the few following differences:

  • The information about the location of each token is structured differently, in a data frame
  • location is gathered for all symbols from the source code, including terminal symbols (tokens), comments
  • An equal sign is identified to be either an assignment, the declaration of a formal argument or the use of an argument

Here is an example file containing R source code that we are going to parse with parser

#' a roxygen comment
f <- function( x = 3 ){
	
	# a regular comment
	rnorm(10 ) + runif( 10 )
	
}

It is a very simple file, for illustration purpose. Let's look what to do with it with the parser package

The parser generates a list of expressions, just like the regular parse function, but the gain is the data attribute. This is a data frame where each token of the parse tree is a line. The id column identifies each line, and the parent column identifies the parent of the current line.

At the moment, only the forthcoming highlight package uses the parser package (actually the parser package has been factored out of highlight), but some anticipated uses of the package include:

  • rework the codetools package so that it tells source location of potential problems
  • code coverage in RUnit or svUnit
  • rework the roxygen parser

Transfer files through Rserve

This post is motivated by this question on R-help. This is a simple java class that sends files through Rserve using the classes RFileInputStream and RFileOutputStream

Then we create a simple file on the client machine:

$ cat > testfile.txt
bla bla
^C

And we are good to go:

$ javac -cp .:REngine.jar:Rserve.jar RserveWire.java
$ java -cp .:REngine.jar:Rserve.jar RserveWire testfile.txt serverfile.txt
/tmp/Rserv/conn6
writing the client file 'testfile.txt' to the server as 'serverfile.txt'
writing the server file 'file.txt' to the client as 'file.txt' 

Now in the directory /tmp/Rserv/conn6 of the server, there are the files "serverfile.txt" and "file.txt"

$ cat serverfile.txt 
bla bla
$ cat file.txt
 [1] -1.16541741 -0.55857285  2.19752036 -0.78432188  1.40739981 -0.87252966
 [7] -0.11545651 -0.36735874 -2.75736666  0.29798096 -0.86836355 -0.03416198
[13] -0.44344089  0.88976360  0.58821334 -0.10354205 -0.88760475 -0.64608338
[19]  0.96552319 -1.57166441 -0.19010633 -1.42239696  0.49363257  0.06167547
[25]  0.34801546 -0.41211734 -0.20320050 -1.45370497  1.34383425 -0.89461504

and on the client there is also the "file.txt"

$ cat file.txt
 [1] -1.16541741 -0.55857285  2.19752036 -0.78432188  1.40739981 -0.87252966
 [7] -0.11545651 -0.36735874 -2.75736666  0.29798096 -0.86836355 -0.03416198
[13] -0.44344089  0.88976360  0.58821334 -0.10354205 -0.88760475 -0.64608338
[19]  0.96552319 -1.57166441 -0.19010633 -1.42239696  0.49363257  0.06167547
[25]  0.34801546 -0.41211734 -0.20320050 -1.45370497  1.34383425 -0.89461504

Monday, August 3 2009

R GUI page on the R wiki

I've started the process of moving the content of this page to the R wiki. The motivation is that the content will become dynamic and updated much more often, people can add their own project, we can have use cases of each gui, tutorials, feature comparison, ...

When we are ready, we will add an entry for the jedit plugin

Tuesday, July 21 2009

RGG#155, 156 and 157

I pushed 3 more graphics from Biecek Przemyslaw to the graphics gallery

A list of popular names for colors from packages RColorBrewer, colorRamps, grDevices

graph_155.png

A set of examples of few graphical low-level parameters lend, ljoin, xpd, adj, legend(), axis, expressions, mtext, srt etc

graph_156.png

Examples for different settings of:

  • the type and width for lines
  • and the type and size for points

graph_157.png

Wednesday, July 8 2009

useR! slides

I've pushed my slides from the presentation I've given at useR! a few minutes ago here

RGG# 154: demo of atomic functions

Przemyslaw Biecek has submitted this graph (and also others I will add later) to the graphics gallery

graph_154.png

A list of examples for the atomic functions polygon(), segments(), symbols(), arrows(), curve(), abline(), points(), lines(). this figure is taken from the book Przewodnik po pakiecie R

Friday, July 3 2009

RGG#153: stars network

Graham Williams sent me this graph, now item 153 in the graph gallery.

graph_153.png

This plot draws a network but at the nodes we have simple segment plots (and in general we might have any other kind of plot at the nodes). This code constructs a network (from the network package), plots it with the nodes being invisibly plotted (using plot.network from the sna package) and then overlays the segment plots using the star function (with the key being located at a position determined by largest.empty from the Hmisc package.

Monday, June 22 2009

using ImageJ from R: the RImageJ package

I've pushed to CRAN the initial version of the RImageJ package. ImageJ is a java based image processing and analysis platform

This version of the package creates an instance of the IJ class as the IJ object in R, so that many methods of this class can be called using the dollar expansion of rJava.

Here is a simple example that uses the package:

download.file( "http://www.google.fr/intl/en_en/images/logo.gif", 
    dest = "google.gif" )

image = IJ$openImage( "google.gif" )
image$show()
IJ$run( "8-bit" )
IJ$run( "Invert" )
IJ$save( "bw-google.gif" )
image$close()


This downloads the google logo, convert it to black and white and inverts it

google logo bw-google.gif Future plans for this package contain:
  • integration of imagej and javagd
  • integration of the imagej macro recorder so that it creates R code instead of code in the imagej macro language

Wednesday, June 17 2009

with semantics for java objects in rJava

I've been playing with the rJava package recently. In a nutshell, rJava lets you create java objects and call methods on them from within the R console

col <- .jnew( "java/awt/Color", 255L, 0L, 0L )
.jcall( col, "I", "getRed" )
# [1] 255
col$getRed()
# [1] 255

The first call uses the regular function .jcall together with the JNI notation to call the getRed method of the created color, the second uses what rJava calls syntactic sugar, so that fields and methods of the object are accessed with the convenient R friendly dollar notation, great !!!

Here, I am just trying to add cream to make the coffee more tasty, by implementing the with method for java object

# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 of the License, or
# (at your option) any later version.
# 
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
# 
# You should have received a copy of the GNU General Public License
# along with this program.  If not, see <http://www.gnu.org/licenses/>.

with.jobjRef <- function( data, expr, ...){
    
    env <- new.env( )
    clazz <- .jcall( data, "Ljava/lang/Class;", "getClass")
    fields <- .jcall( clazz, 
        "[Ljava/lang/reflect/Field;", "getFields" )
    lapply( fields, function(x ){
        n <- .jcall( x, "S", "getName" )
        makeActiveBinding( n, function(v){
            if( missing(v) ){
                # get 
                .jsimplify( .jcall( x, "Ljava/lang/Object;", "get", .jcast( data ) ) )
            } else {
                .jfield( data, n ) <- v
            }
        }, env ) 
    } )
    methods <- .jcall( clazz, 
        "[Ljava/lang/reflect/Method;", "getMethods" )
    lapply( methods, function(m){
        n <- .jcall( m, "S", "getName" )
        assign( n, function(...){
            .jrcall( data, n, ...) 
        }, env = env )
    } )
    assign( "this", data, env = env )
    eval( substitute( expr ), env = env )
}

This allows to call several methods on the same object, for example:

> with( col, {
+   cat( "red = ", getRed(), "\n" )
+   cat( "green = ", getGreen(), "\n" )
+   cat( "blue = ", getBlue(), "\n" )
+ })
red =  255 
green =  0 
blue =  0 
> p <- .jnew("java/awt/Point", 10L, 10L )
> with( p, {
+   move( 40L , 50L )
+   x <- y
+ } )
> p
[1] "Java-Object{java.awt.Point[x=50,y=50]}"

Note in the last example that the x variable that is assigned is the "x" field of the object

Tuesday, June 2 2009

Better completion popups

I've now updated the completion popups used in the advanced editor plugin to get a better display of the available information

For completion items that are related to functions with a help page, we now get this popup

completion-argument.png

When completing arguments of functions, the text of the argument is grabbed from the help page (if available), and displayed as such:

completion-argument.png

For functions that do not have help page, such as functions in the global environment, a deparse of the function is displayed

completion-argument.png

It is probably better to build the plugin from source, but otherwise I have posted a build here

I am currently working on ways to bring this completion mechanism to the console as well

Friday, May 29 2009

biocep's broken REPL

The REPL (Read Eval Print Loop) is the mechanism that R uses to (roughly):

  • get input from the console
  • send output to the console

JRI defines a mechanism for taking advantage of the REPL from java, through the RMainLoopCallbacks interface

biocep takes advantage of this infrastructure by implementing the interface within the DirectJNI class

To circumvent the fact that the REPL is basically an endless loop, gui front-ends usually let the loop run in its own thread, and set this thread to sleep whenever the user does something else than feeding the REPL. This is not quite the way it is done in biocep's implementation. In biocep, when there is no command to run, the REPL thread would wait a short amount of time and then send an empty string "" to the readConsole callback.

The reason why this is deficient is because R might not only use the REPL to ask for top-level command line commands, but also within the environment browser commonly used in R for debugging. Now feeding the browser prompt with an empty command has the effect of stepping out of it, making debugging impossible.

Here is patch against biocep that is a first attempt at solving this issue by implementing the REPL with a sleeping thread, inspired from the way the REPL is implemented in JGR. The patch has been made available to the author of biocep.

Thursday, May 28 2009

xterm256 support in biocep

On this post, I presented the xterm256 package for R, allowing to have text in background and foreground color in the R console

The drawback of relying on xterm escape sequences is that the package needs to be used within a terminal that supports this escape sequence protocol (basically some linux consoles)

Here, I am proposing a patch to the biocep workbench that emulates support for these colors directly in the biocep R console, see the screenshot below:

xterm256-biocep.png

Combined with the syntax highlighter I am working on, this allows syntax highlighting of R code (pretty close to the way R sees the code) directly in the code and is in my view both visually pleasing and very useful

The functionality requires minor modifications of the source code of biocep that I have written and I can send a patch to interested users, but I unfortunately cannot commit the modifications to the biocep project because it is read-only at the moment

Wednesday, May 20 2009

Disable specific warnings

[This post is inspired from this thread on R-help]

The suppressWarnings function allows to call a function and suppress whatever warning it might generate. This uses the calling handler and restart mechanism of R to invoke the special "muffleWarning" restart when a warning is detected by the handler.

The downside is that it removes all warnings, and this might not be what you want. Consider that simple function that gives two warnings:

f <- function( x) {
  warning( "bla bla" )
  y <- x + 3
  warning( "yada yada" )
  y
}

> f(5)
[1] 8
Warning messages:
1: In f(5) : bla bla
2: In f(5) : yada yada
> suppressWarnings( f(5) )
[1] 8

What if I wanted to remove "bla bla" warnings and not "yada yada" warnings, because I know that "bla bla" warnings are expected and are more disturbing that useful. Currently, suppressWarnings does not offer that possibility, but you can make you own calling handler that handles warnings the way you want:

> h <- function(w) if( any( grepl( "bla", w) ) ) invokeRestart( "muffleWarning" )
> withCallingHandlers( f(5), warning = h )
[1] 8
Warning message:
In f(5) : yada yada

Thursday, May 7 2009

Search for R mailing list archives with gmane

This thread on R-devel led me to write these functions to search into R mailing list archives through gmane.
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# 
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
# 
# You should have received a copy of the GNU General Public License
# along with this program.  If not, see <http://www.gnu.org/licenses/>.

#' search on gmane.org mailing list archives
#'
#' @param string query
#' @param group search on which gmane group
#' @param author author of the email
#' @param sort sort order criteria
#' @param op how to combine words
#' @param r.group R related group to use, the default "*" means all r related groups
gmaneSearch <- function( string, 
    group = paste( "gmane.comp.lang.r.", r.group, sep = ""), 
    author = "", sort = c("relevance", "date", "revdate"), 
    op = c("and", "or"), 
    r.group = "*" ){
    
    sort <- match.arg(sort)
    op <- match.arg( op )
    
    url <- sprintf( 
        'http://search.gmane.org/?query=%s&author=%s&group=%s&sort=%s&DEFAULTOP=%s', 
            gsub( ' +', '+', string),  author,  group,  sort, op )
    url <- URLencode( url )
    browseURL( url )
}

#' retrieves the list of gmane groups associated with a prefix
#' 
#' @param prefix group prefix
gmaneGroups <- function( prefix = "gmane.comp.lang.r." ){
    url <- URLencode( sprintf( "http://dir.gmane.org/index.php?prefix=%s", prefix) )
    txt <- grep( '^<tr.*<td align=right.*<a', readLines( url ), value = TRUE )
    
    rx <- '^.*?<a href="(.*?)">(.*?)</a>.*<td>(.*?)</td>.*$'
    out <- data.frame( 
        url = gsub( rx, "\\1", txt ), 
        group = gsub( rx, "\\2", txt ),
        description = gsub( rx, "\\3", txt ), 
        stringsAsFactors = FALSE
        )
    out$group <- sub( "...", ".*", out$group, fixed = TRUE )
    out
}

So for example:

R> gmaneSearch ("browser prompt", author="romain" )
pops up this page

Saturday, April 18 2009

Colorful terminal: the R package "xterm256"

One of the goal of my forthcoming highlight package for R is to provide syntax highlighting based on evidence gathered from the output of the R parser directly into the R console (more on this later)

While writing the renderer targetting the console, I realized that support for colored text in the console is something that might be useful outside of the highlighter, and then decided to make it an independent package : xterm256

The idea is to use the The 256 color mode of xterm to wrap some text between escape sequences so that when it is cat to the console, the text appears with a background and/or a foreground color. Here is a screenshot from my console: lef-2.png

The package exposes only one function with three arguments: the style function with arguments :

  • x: the text we want to style
  • bg: the background color we want to use
  • fg: the foreground color we want to use

so if you want to print hello world in yellow with a black background, you can do:

cat( style( "hello world", bg = "black", fg = "yellow"), "\n" )

Friday, March 27 2009

Document java software with "R CMD build", ant, and javadoc

This is the sequel of this post where we look at ways to build javadoc documentation as part of an R package

Ant has the javadoc task which we will use to build html files from java sources, leading to this modified build script with the new javadoc target:

   1 <project name="Hello Java World" basedir="." default="build" >
   2 
   3   <property name="target.dir" value="../inst/java" />
   4   <property name="javadoc.dir" value="../inst/javadoc" />
   5   
   6   <target name="clean">
   7     <delete dir="bin" />
   8     <delete dir="${javadoc.dir}" />
   9   </target>
  10   
  11   <target name="compile">
  12     <mkdir dir="bin"/>
  13     <javac srcdir="src" destdir="bin" />
  14   </target>
  15   
  16   <target name="javadoc">
  17     <mkdir dir="${javadoc.dir}" />
  18     <javadoc access="public" destdir="${javadoc.dir}"
  19       author="true" version="true" use="true" 
  20       windowtitle="helloJavaWorld - Java API">
  21         
  22        <fileset dir="src" defaultexcludes="yes">
  23             <include name="**/*.java"/>
  24         </fileset>
  25           
  26     </javadoc>
  27   </target>
  28   
  29   <target name="build" depends="compile,javadoc">
  30     <jar jarfile="${target.dir}/hellojavaworld.jar">
  31       <fileset dir="bin" />
  32     </jar>
  33   </target>
  34   
  35 </project>
  36 

so that when we run "R CMD build", the directory inst/javadoc gets created and filled with documentation generated from the java sources

The next thing we need is a way to access this documentation from R documentation, so we will just add a link to it from the Rd file, I have added this into the helloJavaWorld.Rd file in the seealso section:

18 \seealso{\link[helloJavaWorld:../javadoc/index]{The Java API of the package}}

This is not a perfect solution but gives a starting point. Potential issues, things to improve :

  • It would be better to have a link in the 00Index.html file, but there currently is no way to do something like that
  • A way to jump back to the R documentation from the java documentation is needed

I have posted a copy of the built package here, and you can download the source of this version here

Next step, use the junitreport ant task to make a report of unit tests of the java code

Thursday, March 26 2009

update on RGG#152

Taiyun Wei has updated RGG#152 with possibility to reorder the correlation matrix using PCA and choice of colors

graph_152__2.png

See Taiyun blog for a nice picasa animation

- page 5 of 7 -